Hey Jeremy, Thanks for your feedback - I agree search needs to performant. Im glad you're also looking at these issues!
A priority for me is the inclusion of *fuzzy* searching - a quick look seems lunr does support it: https://lunrjs.com/guides/searching.html#fuzzy-matches but elastic does not. Best, Diego On Wednesday, June 27, 2018 at 2:28:18 AM UTC-5, Jeremy Ruston wrote: > > Hi Diego > > Filter operators are often re-executed and so it's important that they > perform well. For an expensive operation like a search we need to try to > cache as much of the work as we can. > > An approach that should work is to introduce a new startup module that > tracks changes to the store in order to keep the index up to date, and then > have the filter operator call a version of the search function that caches > the results (and clears the cache when it is invalidated by changes to the > tiddler store). > > Having said that, one of the client projects I'm working on for Federatial > looks like it's going to need an enhanced search capability. If the project > proceeds to the next phase then I'll be looking at adding a third party > search index library -- I've been looking at https://lunrjs.com/ and > http://elasticlunr.com/. The API that they feature allows the index to be > dynamically modified by adding and removing data incrementally; fuse.js > looks like it only accepts a monolithic block of data, and would require a > re-index each time the tiddler store changes. > > Best wishes > > Jeremy > > -- > Jeremy Ruston > jer...@jermolene.com <javascript:> > https://jermolene.com > > On 26 Jun 2018, at 20:45, Diego Mesa <dieg...@gmail.com <javascript:>> > wrote: > > Hey all, > > Thanks Jeremy for sending that. The following is *more pseduo-code than > anything*, but reflects my quick hacking: > > (function(){ > > > /*jslint node: true, browser: true */ > /*global $tw: false */ > "use strict"; > > > var fuze = require("$:/plugins/fuzzy/fuze.js"); > > > var options = { > shouldSort: true, > matchAllTokens: true, > threshold: 0.6, > location: 0, > distance: 100, > maxPatternLength: 32, > minMatchCharLength: 1, > keys: undefined > }; > > > /* > Export our filter function > */ > exports.fuzzysearch = function(source,operator,options) { > var results = []; > > > source = options.source || $tw.wiki.each; > > > var searchTiddler = function(title){ > jsonData = $tw.wiki.getTiddlerAsJson(title) > var fuse = new Fuse(jsonData, options); // "list" is the item > array > var result = fuse.search(operator.operand); > return result; > }; > source(function(tiddler,title) { > if(searchTiddler(title) !== options.invert) { > results.push(title); > } > }); > > > console.log("e"); > return results; > }; > > > })(); > > I quickly generated this just by looking at: > > $:/core/modules/filters/search.js > > which is just a wrapper around: > > $:/core/modules/wiki.js > > where exports.search = is defined. > > The issue is that as you can see from > > http://fusejs.io/ > > Fuze works better if it would have access to the entire json string of the > wiki, and it can handle the searching and scoring itself. > > What do you guys think? Any tips to help me move forward? > > Best, > Diego > > On Tuesday, June 26, 2018 at 8:38:36 AM UTC-5, Jed Carty wrote: >> >> I hadn't realised it could be done that way. That makes including >> external libraries a lot easier than I thought. I am a bit embarrassed that >> I didn't notice you could do that. >> > -- > You received this message because you are subscribed to the Google Groups > "TiddlyWikiDev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to tiddlywikide...@googlegroups.com <javascript:>. > To post to this group, send email to tiddly...@googlegroups.com > <javascript:>. > Visit this group at https://groups.google.com/group/tiddlywikidev. > To view this discussion on the web visit > https://groups.google.com/d/msgid/tiddlywikidev/c0e08271-5dd4-4578-b2b3-d4693c27a8b4%40googlegroups.com > > <https://groups.google.com/d/msgid/tiddlywikidev/c0e08271-5dd4-4578-b2b3-d4693c27a8b4%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group. To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikidev+unsubscr...@googlegroups.com. To post to this group, send email to tiddlywikidev@googlegroups.com. Visit this group at https://groups.google.com/group/tiddlywikidev. To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/aab84e80-923e-4395-8387-604b23a3db61%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
