Hey Jeremy, Just wanted to let you know of this plugin:
https://github.com/hoelzro/tw-full-text-search which incorporates lunr.js into TW for full text searching. I just tried today and am somewhat able to do fuzzy searching with it (does not deal with immediatel searching properly yet). I have let him know about this discussion and the corresponding feature request on github, and have opened an issue on his repo for the fuzzy matching: https://github.com/hoelzro/tw-full-text-search/issues/4 Perhaps you can reuse some of hoelzro's work? Best, Diego On Wednesday, June 27, 2018 at 8:47:06 AM UTC-5, Diego Mesa wrote: > > Hey Jeremy, > > Thanks for your feedback - I agree search needs to performant. Im glad > you're also looking at these issues! > > A priority for me is the inclusion of *fuzzy* searching - a quick look > seems lunr does support it: > > https://lunrjs.com/guides/searching.html#fuzzy-matches > > but elastic does not. > > Best, > Diego > > On Wednesday, June 27, 2018 at 2:28:18 AM UTC-5, Jeremy Ruston wrote: >> >> Hi Diego >> >> Filter operators are often re-executed and so it's important that they >> perform well. For an expensive operation like a search we need to try to >> cache as much of the work as we can. >> >> An approach that should work is to introduce a new startup module that >> tracks changes to the store in order to keep the index up to date, and then >> have the filter operator call a version of the search function that caches >> the results (and clears the cache when it is invalidated by changes to the >> tiddler store). >> >> Having said that, one of the client projects I'm working on for >> Federatial looks like it's going to need an enhanced search capability. If >> the project proceeds to the next phase then I'll be looking at adding a >> third party search index library -- I've been looking at >> https://lunrjs.com/ and http://elasticlunr.com/. The API that they >> feature allows the index to be dynamically modified by adding and removing >> data incrementally; fuse.js looks like it only accepts a monolithic block >> of data, and would require a re-index each time the tiddler store changes. >> >> Best wishes >> >> Jeremy >> >> -- >> Jeremy Ruston >> [email protected] >> https://jermolene.com >> >> On 26 Jun 2018, at 20:45, Diego Mesa <[email protected]> wrote: >> >> Hey all, >> >> Thanks Jeremy for sending that. The following is *more pseduo-code than >> anything*, but reflects my quick hacking: >> >> (function(){ >> >> >> /*jslint node: true, browser: true */ >> /*global $tw: false */ >> "use strict"; >> >> >> var fuze = require("$:/plugins/fuzzy/fuze.js"); >> >> >> var options = { >> shouldSort: true, >> matchAllTokens: true, >> threshold: 0.6, >> location: 0, >> distance: 100, >> maxPatternLength: 32, >> minMatchCharLength: 1, >> keys: undefined >> }; >> >> >> /* >> Export our filter function >> */ >> exports.fuzzysearch = function(source,operator,options) { >> var results = []; >> >> >> source = options.source || $tw.wiki.each; >> >> >> var searchTiddler = function(title){ >> jsonData = $tw.wiki.getTiddlerAsJson(title) >> var fuse = new Fuse(jsonData, options); // "list" is the item >> array >> var result = fuse.search(operator.operand); >> return result; >> }; >> source(function(tiddler,title) { >> if(searchTiddler(title) !== options.invert) { >> results.push(title); >> } >> }); >> >> >> console.log("e"); >> return results; >> }; >> >> >> })(); >> >> I quickly generated this just by looking at: >> >> $:/core/modules/filters/search.js >> >> which is just a wrapper around: >> >> $:/core/modules/wiki.js >> >> where exports.search = is defined. >> >> The issue is that as you can see from >> >> http://fusejs.io/ >> >> Fuze works better if it would have access to the entire json string of >> the wiki, and it can handle the searching and scoring itself. >> >> What do you guys think? Any tips to help me move forward? >> >> Best, >> Diego >> >> On Tuesday, June 26, 2018 at 8:38:36 AM UTC-5, Jed Carty wrote: >>> >>> I hadn't realised it could be done that way. That makes including >>> external libraries a lot easier than I thought. I am a bit embarrassed that >>> I didn't notice you could do that. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "TiddlyWikiDev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/tiddlywikidev. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/tiddlywikidev/c0e08271-5dd4-4578-b2b3-d4693c27a8b4%40googlegroups.com >> >> <https://groups.google.com/d/msgid/tiddlywikidev/c0e08271-5dd4-4578-b2b3-d4693c27a8b4%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> >> -- You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/tiddlywikidev. To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/7667efe5-274c-4eb7-9eb9-f6ca3da3769a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
