Hi Diego Filter operators are often re-executed and so it's important that they perform well. For an expensive operation like a search we need to try to cache as much of the work as we can.
An approach that should work is to introduce a new startup module that tracks changes to the store in order to keep the index up to date, and then have the filter operator call a version of the search function that caches the results (and clears the cache when it is invalidated by changes to the tiddler store). Having said that, one of the client projects I'm working on for Federatial looks like it's going to need an enhanced search capability. If the project proceeds to the next phase then I'll be looking at adding a third party search index library -- I've been looking at https://lunrjs.com/ and http://elasticlunr.com/. The API that they feature allows the index to be dynamically modified by adding and removing data incrementally; fuse.js looks like it only accepts a monolithic block of data, and would require a re-index each time the tiddler store changes. Best wishes Jeremy -- Jeremy Ruston [email protected] https://jermolene.com > On 26 Jun 2018, at 20:45, Diego Mesa <[email protected]> wrote: > > Hey all, > > Thanks Jeremy for sending that. The following is more pseduo-code than > anything, but reflects my quick hacking: > > (function(){ > > > /*jslint node: true, browser: true */ > /*global $tw: false */ > "use strict"; > > > var fuze = require("$:/plugins/fuzzy/fuze.js"); > > > var options = { > shouldSort: true, > matchAllTokens: true, > threshold: 0.6, > location: 0, > distance: 100, > maxPatternLength: 32, > minMatchCharLength: 1, > keys: undefined > }; > > > /* > Export our filter function > */ > exports.fuzzysearch = function(source,operator,options) { > var results = []; > > > source = options.source || $tw.wiki.each; > > > var searchTiddler = function(title){ > jsonData = $tw.wiki.getTiddlerAsJson(title) > var fuse = new Fuse(jsonData, options); // "list" is the item array > var result = fuse.search(operator.operand); > return result; > }; > source(function(tiddler,title) { > if(searchTiddler(title) !== options.invert) { > results.push(title); > } > }); > > > console.log("e"); > return results; > }; > > > })(); > > I quickly generated this just by looking at: > > $:/core/modules/filters/search.js > > which is just a wrapper around: > > $:/core/modules/wiki.js > > where exports.search = is defined. > > The issue is that as you can see from > > http://fusejs.io/ > > Fuze works better if it would have access to the entire json string of the > wiki, and it can handle the searching and scoring itself. > > What do you guys think? Any tips to help me move forward? > > Best, > Diego > >> On Tuesday, June 26, 2018 at 8:38:36 AM UTC-5, Jed Carty wrote: >> I hadn't realised it could be done that way. That makes including external >> libraries a lot easier than I thought. I am a bit embarrassed that I didn't >> notice you could do that. > > -- > You received this message because you are subscribed to the Google Groups > "TiddlyWikiDev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/tiddlywikidev. > To view this discussion on the web visit > https://groups.google.com/d/msgid/tiddlywikidev/c0e08271-5dd4-4578-b2b3-d4693c27a8b4%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/tiddlywikidev. To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/193465C7-43F2-4CD2-8DBF-02593D190C36%40gmail.com. For more options, visit https://groups.google.com/d/optout.
