Re: Search while typing (incremental search)
I have added a QnA https://cwiki.apache.org/confluence/display/LUCENE/LuceneFAQ#LuceneFAQ-DoesLucenesupportauto-suggest/autocomplete? I will also try to provide an example, for example https://medium.com/@ekaterinamihailova/in-memory-search-and-autocomplete-with-lucene-8-5-f2df1bc71c36 https://github.com/Da-Bulgaria/e-prescriptions/blob/master/src/main/java/bg/ehealth/prescriptions/services/icd/ICDService.java or https://stackoverflow.com/questions/24968697/how-to-implement-auto-suggest-using-lucenes-new-analyzinginfixsuggester-api but first need to check whether these examples are according to Lucene 8.10.1 suggest API If you know any simple, recent examples, please let me know Thanks Michael Am 08.10.21 um 21:40 schrieb Michael Wechner: Am 08.10.21 um 18:49 schrieb Michael Sokolov: Thank you for offering to add to the FAQ! Indeed it should mention the suggester capability. I think you have permissions to edit that wiki? yes :-) Please go ahead and I think add a link to the suggest module javadocs ok, will do! Thanks Michael On Thu, Oct 7, 2021 at 2:30 AM Michael Wechner wrote: Thanks very much for your feedback! I will try it :-) As I wrote I would like to add a summary to the Lucene FAQ (https://cwiki.apache.org/confluence/display/lucene/lucenefaq) Would the following questions make sense? - "Does Lucene support incremental search?" - "Does Lucene support auto completion suggestions?" Or would other other terms / or another wording make more sense? Thanks Michael Am 07.10.21 um 01:14 schrieb Robert Muir: TLDR: use the lucene suggest/ package. Start with building suggester from your query logs (either a file or index them). These have a lot of flexibility about how the matches happen, for example pure prefixes, edit distance typos, infix matching, analysis chain, even now Japanese input-method integration :) Run that suggester on the user input, retrieving say, the top 5-10 matches of relevant query suggestions. return those in the UI (typical autosuggest-type field), but also run a search on the first one. The user gets the instant-search experience, but when they type 'tes', you search on 'tesla' (if that's the top-suggested query, the highlighted one in the autocomplete). if they arrow-down to another suggestion such as 'test' or type a 't' or use the mouse or whatever, then the process runs again and they see the results for that. IMO for most cases this leads to a saner experience than trying to rank all documents based on a prefix 'tes': the problem is there is still too much query ambiguity, not really any "keywords" yet, so trying to rank those documents won't be very useful. Instead you try to "interact" with the user to present results in a useful way that they can navigate. On the other hand if you really want to just search on prefixes and jumble up the results (perhaps because you are gonna just sort by some custom document feature instead of relevance), then you can do that if you really want. You can use the n-gram/edge-ngram/shingle filters in the analysis package for that. On Wed, Oct 6, 2021 at 5:37 PM Michael Wechner wrote: Hi I am trying to implement a search with Lucene similar to what for example various "Note Apps" (e.g. "Google Keep" or "Samsung Notes") are offering, that with every new letter typed a new search is being executed. For example when I type "tes", then all documents are being returned containing the word "test" or "tesla" and when I continue typing, for example "tesö" and there are no documents containing the string "tesö", then the app will tell me that there are no matches. I have found a couple of articles related to this kind of search, for example https://stackoverflow.com/questions/10828825/incremental-search-using-lucene https://stackoverflow.com/questions/120180/how-to-do-query-auto-completion-suggestions-in-lucene but would be great to know whether there exist other possibilities or what the best practice is? I am even not sure what the right term for this kind of search is, is it really "incremental search" or something else? Looking forward to your feedback and will be happy to extend the Lucene FAQ once I understand better :-) Thanks Michael - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail:
Re: Search while typing (incremental search)
Am 08.10.21 um 18:49 schrieb Michael Sokolov: Thank you for offering to add to the FAQ! Indeed it should mention the suggester capability. I think you have permissions to edit that wiki? yes :-) Please go ahead and I think add a link to the suggest module javadocs ok, will do! Thanks Michael On Thu, Oct 7, 2021 at 2:30 AM Michael Wechner wrote: Thanks very much for your feedback! I will try it :-) As I wrote I would like to add a summary to the Lucene FAQ (https://cwiki.apache.org/confluence/display/lucene/lucenefaq) Would the following questions make sense? - "Does Lucene support incremental search?" - "Does Lucene support auto completion suggestions?" Or would other other terms / or another wording make more sense? Thanks Michael Am 07.10.21 um 01:14 schrieb Robert Muir: TLDR: use the lucene suggest/ package. Start with building suggester from your query logs (either a file or index them). These have a lot of flexibility about how the matches happen, for example pure prefixes, edit distance typos, infix matching, analysis chain, even now Japanese input-method integration :) Run that suggester on the user input, retrieving say, the top 5-10 matches of relevant query suggestions. return those in the UI (typical autosuggest-type field), but also run a search on the first one. The user gets the instant-search experience, but when they type 'tes', you search on 'tesla' (if that's the top-suggested query, the highlighted one in the autocomplete). if they arrow-down to another suggestion such as 'test' or type a 't' or use the mouse or whatever, then the process runs again and they see the results for that. IMO for most cases this leads to a saner experience than trying to rank all documents based on a prefix 'tes': the problem is there is still too much query ambiguity, not really any "keywords" yet, so trying to rank those documents won't be very useful. Instead you try to "interact" with the user to present results in a useful way that they can navigate. On the other hand if you really want to just search on prefixes and jumble up the results (perhaps because you are gonna just sort by some custom document feature instead of relevance), then you can do that if you really want. You can use the n-gram/edge-ngram/shingle filters in the analysis package for that. On Wed, Oct 6, 2021 at 5:37 PM Michael Wechner wrote: Hi I am trying to implement a search with Lucene similar to what for example various "Note Apps" (e.g. "Google Keep" or "Samsung Notes") are offering, that with every new letter typed a new search is being executed. For example when I type "tes", then all documents are being returned containing the word "test" or "tesla" and when I continue typing, for example "tesö" and there are no documents containing the string "tesö", then the app will tell me that there are no matches. I have found a couple of articles related to this kind of search, for example https://stackoverflow.com/questions/10828825/incremental-search-using-lucene https://stackoverflow.com/questions/120180/how-to-do-query-auto-completion-suggestions-in-lucene but would be great to know whether there exist other possibilities or what the best practice is? I am even not sure what the right term for this kind of search is, is it really "incremental search" or something else? Looking forward to your feedback and will be happy to extend the Lucene FAQ once I understand better :-) Thanks Michael - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: Search while typing (incremental search)
Thank you for offering to add to the FAQ! Indeed it should mention the suggester capability. I think you have permissions to edit that wiki? Please go ahead and I think add a link to the suggest module javadocs On Thu, Oct 7, 2021 at 2:30 AM Michael Wechner wrote: > > Thanks very much for your feedback! > > I will try it :-) > > As I wrote I would like to add a summary to the Lucene FAQ > (https://cwiki.apache.org/confluence/display/lucene/lucenefaq) > > Would the following questions make sense? > > - "Does Lucene support incremental search?" > > - "Does Lucene support auto completion suggestions?" > > Or would other other terms / or another wording make more sense? > > Thanks > > Michael > > > > Am 07.10.21 um 01:14 schrieb Robert Muir: > > TLDR: use the lucene suggest/ package. Start with building suggester > > from your query logs (either a file or index them). > > These have a lot of flexibility about how the matches happen, for > > example pure prefixes, edit distance typos, infix matching, analysis > > chain, even now Japanese input-method integration :) > > > > Run that suggester on the user input, retrieving say, the top 5-10 > > matches of relevant query suggestions. > > return those in the UI (typical autosuggest-type field), but also run > > a search on the first one. > > > > The user gets the instant-search experience, but when they type 'tes', > > you search on 'tesla' (if that's the top-suggested query, the > > highlighted one in the autocomplete). if they arrow-down to another > > suggestion such as 'test' or type a 't' or use the mouse or whatever, > > then the process runs again and they see the results for that. > > > > IMO for most cases this leads to a saner experience than trying to > > rank all documents based on a prefix 'tes': the problem is there is > > still too much query ambiguity, not really any "keywords" yet, so > > trying to rank those documents won't be very useful. Instead you try > > to "interact" with the user to present results in a useful way that > > they can navigate. > > > > On the other hand if you really want to just search on prefixes and > > jumble up the results (perhaps because you are gonna just sort by some > > custom document feature instead of relevance), then you can do that if > > you really want. You can use the n-gram/edge-ngram/shingle filters in > > the analysis package for that. > > > > On Wed, Oct 6, 2021 at 5:37 PM Michael Wechner > > wrote: > >> Hi > >> > >> I am trying to implement a search with Lucene similar to what for > >> example various "Note Apps" (e.g. "Google Keep" or "Samsung Notes") are > >> offering, that with every new letter typed a new search is being executed. > >> > >> For example when I type "tes", then all documents are being returned > >> containing the word "test" or "tesla" and when I continue typing, for > >> example "tesö" and there are no documents containing the string "tesö", > >> then the app will tell me that there are no matches. > >> > >> I have found a couple of articles related to this kind of search, for > >> example > >> > >> https://stackoverflow.com/questions/10828825/incremental-search-using-lucene > >> > >> https://stackoverflow.com/questions/120180/how-to-do-query-auto-completion-suggestions-in-lucene > >> > >> but would be great to know whether there exist other possibilities or > >> what the best practice is? > >> > >> I am even not sure what the right term for this kind of search is, is it > >> really "incremental search" or something else? > >> > >> Looking forward to your feedback and will be happy to extend the Lucene > >> FAQ once I understand better :-) > >> > >> Thanks > >> > >> Michael > >> > >> - > >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > >> For additional commands, e-mail: java-user-h...@lucene.apache.org > >> > > - > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > - > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: Search while typing (incremental search)
Thanks very much for your feedback! I will try it :-) As I wrote I would like to add a summary to the Lucene FAQ (https://cwiki.apache.org/confluence/display/lucene/lucenefaq) Would the following questions make sense? - "Does Lucene support incremental search?" - "Does Lucene support auto completion suggestions?" Or would other other terms / or another wording make more sense? Thanks Michael Am 07.10.21 um 01:14 schrieb Robert Muir: TLDR: use the lucene suggest/ package. Start with building suggester from your query logs (either a file or index them). These have a lot of flexibility about how the matches happen, for example pure prefixes, edit distance typos, infix matching, analysis chain, even now Japanese input-method integration :) Run that suggester on the user input, retrieving say, the top 5-10 matches of relevant query suggestions. return those in the UI (typical autosuggest-type field), but also run a search on the first one. The user gets the instant-search experience, but when they type 'tes', you search on 'tesla' (if that's the top-suggested query, the highlighted one in the autocomplete). if they arrow-down to another suggestion such as 'test' or type a 't' or use the mouse or whatever, then the process runs again and they see the results for that. IMO for most cases this leads to a saner experience than trying to rank all documents based on a prefix 'tes': the problem is there is still too much query ambiguity, not really any "keywords" yet, so trying to rank those documents won't be very useful. Instead you try to "interact" with the user to present results in a useful way that they can navigate. On the other hand if you really want to just search on prefixes and jumble up the results (perhaps because you are gonna just sort by some custom document feature instead of relevance), then you can do that if you really want. You can use the n-gram/edge-ngram/shingle filters in the analysis package for that. On Wed, Oct 6, 2021 at 5:37 PM Michael Wechner wrote: Hi I am trying to implement a search with Lucene similar to what for example various "Note Apps" (e.g. "Google Keep" or "Samsung Notes") are offering, that with every new letter typed a new search is being executed. For example when I type "tes", then all documents are being returned containing the word "test" or "tesla" and when I continue typing, for example "tesö" and there are no documents containing the string "tesö", then the app will tell me that there are no matches. I have found a couple of articles related to this kind of search, for example https://stackoverflow.com/questions/10828825/incremental-search-using-lucene https://stackoverflow.com/questions/120180/how-to-do-query-auto-completion-suggestions-in-lucene but would be great to know whether there exist other possibilities or what the best practice is? I am even not sure what the right term for this kind of search is, is it really "incremental search" or something else? Looking forward to your feedback and will be happy to extend the Lucene FAQ once I understand better :-) Thanks Michael - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: Search while typing (incremental search)
TLDR: use the lucene suggest/ package. Start with building suggester from your query logs (either a file or index them). These have a lot of flexibility about how the matches happen, for example pure prefixes, edit distance typos, infix matching, analysis chain, even now Japanese input-method integration :) Run that suggester on the user input, retrieving say, the top 5-10 matches of relevant query suggestions. return those in the UI (typical autosuggest-type field), but also run a search on the first one. The user gets the instant-search experience, but when they type 'tes', you search on 'tesla' (if that's the top-suggested query, the highlighted one in the autocomplete). if they arrow-down to another suggestion such as 'test' or type a 't' or use the mouse or whatever, then the process runs again and they see the results for that. IMO for most cases this leads to a saner experience than trying to rank all documents based on a prefix 'tes': the problem is there is still too much query ambiguity, not really any "keywords" yet, so trying to rank those documents won't be very useful. Instead you try to "interact" with the user to present results in a useful way that they can navigate. On the other hand if you really want to just search on prefixes and jumble up the results (perhaps because you are gonna just sort by some custom document feature instead of relevance), then you can do that if you really want. You can use the n-gram/edge-ngram/shingle filters in the analysis package for that. On Wed, Oct 6, 2021 at 5:37 PM Michael Wechner wrote: > > Hi > > I am trying to implement a search with Lucene similar to what for > example various "Note Apps" (e.g. "Google Keep" or "Samsung Notes") are > offering, that with every new letter typed a new search is being executed. > > For example when I type "tes", then all documents are being returned > containing the word "test" or "tesla" and when I continue typing, for > example "tesö" and there are no documents containing the string "tesö", > then the app will tell me that there are no matches. > > I have found a couple of articles related to this kind of search, for > example > > https://stackoverflow.com/questions/10828825/incremental-search-using-lucene > > https://stackoverflow.com/questions/120180/how-to-do-query-auto-completion-suggestions-in-lucene > > but would be great to know whether there exist other possibilities or > what the best practice is? > > I am even not sure what the right term for this kind of search is, is it > really "incremental search" or something else? > > Looking forward to your feedback and will be happy to extend the Lucene > FAQ once I understand better :-) > > Thanks > > Michael > > - > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org