Re: Solr Auto-Complete
Thanks Alexandre. I think it is clear. On Sun, Dec 6, 2015 at 5:21 PM, Alexandre Rafalovitchwrote: > For suffix matches, you copy text the field and in the different type add > string reversal for both index and query portions. So you are doing prefix > matching algorithm but on reversed strings. > > I can dig up an example if it is not clear. > On 6 Dec 2015 8:06 am, "Salman Ansari" wrote: > > > That is right. I am actually looking for phrase prefixes not each term > > prefix within the phrase. That satisfies my requirements. However, my > > additional question was how do I manipulate the filedType to later allow > > for suffix matches as well? or will that be a completely different > > fieldType definition? > > > > Regards, > > Salman > > > > > > On Sun, Dec 6, 2015 at 2:12 PM, Andrea Gazzarini > > wrote: > > > > > Sorry, my damned mobile: "Is that close to what you were looking for?" > > > > > > 2015-12-06 12:07 GMT+01:00 Andrea Gazzarini : > > > > > > > Do you mean "phrase" or "term" prefixes? If you try to put a field > > value > > > > (two or more terms) in the analysis page you will see what the index > > > > analyzer chain (of my example field type) is doing. The whole value > is > > > > managed as a single-ngrammed token, so you will get only a phrase > > prefix > > > > search, as in your request. > > > > > > > > If you want to manage also terms prefixes, I would also index another > > > > field (similar to the example you posted); then, the search handler > > with > > > > e(dismax) would have something like this: > > > > > > > > > > > >> > > > > text_suggestion_phrase_prefix_search^b1 > > > > text_suggestion_terms_prefix_search^b2 > > > > > > > > > > > > > > > > > > > > b1 and b2 values strictly depend on your search logic. > > > > > > > > Is that close that what you were looking for? > > > > > > > > Best, > > > > Andrea > > > > > > > > > > > > > > > > 2015-12-06 11:53 GMT+01:00 Salman Ansari : > > > > > > > >> Thanks a lot Andrea. It did work. > > > >> > > > >> However, just for my understanding, can you please explain more how > > did > > > >> you > > > >> make it work for prefixes. I know you mentioned using another > > Tokenizer > > > >> but > > > >> for example, if I want to tweak it later on to work on suffixes or > > > within > > > >> phrases how should I go about that? > > > >> > > > >> Thanks again for your help. > > > >> > > > >> Regards, > > > >> Salman > > > >> > > > >> > > > >> On Sun, Dec 6, 2015 at 1:24 PM, Andrea Gazzarini < > > a.gazzar...@gmail.com > > > > > > > >> wrote: > > > >> > > > >> > Hi Salman, > > > >> > that's because you're using a StandardTokenizer. Try with > something > > > like > > > >> > this (copied, pasted and changed using my phone so probably with a > > lot > > > >> of > > > >> > mistakes ;) but you should be able to get what I mean). BTW I > don't > > > >> know if > > > >> > that's the case but I would also put a MappingCharFilterFactory > > > >> > > > > >> > > > >> > positionIncrementGap="100"> > > > >> > > > > >> > * > > >> > mapping="mapping-FoldToASCII.txt"/>* > > > >> > > > > >> > > > > >> > > > >> > generateWordParts="0" generateNumberParts="0" catenateAll="1" > > > >> > splitOnCaseChange="0" /> > > > >> > > > >> > maxGramSize="20"/> > > > >> > > > > >> > > > > >> > * > > >> > mapping="mapping-FoldToASCII.txt"/>* > > > >> > > > > >> > > > > >> > > > >> > generateWordParts="0" generateNumberParts="0" catenateAll="1" > > > >> > splitOnCaseChange="0" /> > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > 2015-12-06 9:36 GMT+01:00 Salman Ansari >: > > > >> > > > > >> > > Hi, > > > >> > > > > > >> > > > > > >> > > > > > >> > > I have updated my schema.xml as mentioned in the previous posts > > > using > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > positionIncrementGap="100"> > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> minGramSize="1" > > > >> > > maxGramSize="20"/> > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > This does the auto-complete, but it does it at every portion of > > the > > > >> text > > > >> > > (not just at the beginning) (prefix). So searching for "And" in > my > > > >> field > > > >> > > for locations returns both of the following documents. > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > 1 > > > >> > > > > > >> > > AD > > > >> > > > > > >> > > *And*orra > > > >> > > > > > >> > > أندورا > > > >> > > > > >
Re: Solr Auto-Complete
For suffix matches, you copy text the field and in the different type add string reversal for both index and query portions. So you are doing prefix matching algorithm but on reversed strings. I can dig up an example if it is not clear. On 6 Dec 2015 8:06 am, "Salman Ansari"wrote: > That is right. I am actually looking for phrase prefixes not each term > prefix within the phrase. That satisfies my requirements. However, my > additional question was how do I manipulate the filedType to later allow > for suffix matches as well? or will that be a completely different > fieldType definition? > > Regards, > Salman > > > On Sun, Dec 6, 2015 at 2:12 PM, Andrea Gazzarini > wrote: > > > Sorry, my damned mobile: "Is that close to what you were looking for?" > > > > 2015-12-06 12:07 GMT+01:00 Andrea Gazzarini : > > > > > Do you mean "phrase" or "term" prefixes? If you try to put a field > value > > > (two or more terms) in the analysis page you will see what the index > > > analyzer chain (of my example field type) is doing. The whole value is > > > managed as a single-ngrammed token, so you will get only a phrase > prefix > > > search, as in your request. > > > > > > If you want to manage also terms prefixes, I would also index another > > > field (similar to the example you posted); then, the search handler > with > > > e(dismax) would have something like this: > > > > > > > > >> > > > text_suggestion_phrase_prefix_search^b1 > > > text_suggestion_terms_prefix_search^b2 > > > > > > > > > > > > > > > b1 and b2 values strictly depend on your search logic. > > > > > > Is that close that what you were looking for? > > > > > > Best, > > > Andrea > > > > > > > > > > > > 2015-12-06 11:53 GMT+01:00 Salman Ansari : > > > > > >> Thanks a lot Andrea. It did work. > > >> > > >> However, just for my understanding, can you please explain more how > did > > >> you > > >> make it work for prefixes. I know you mentioned using another > Tokenizer > > >> but > > >> for example, if I want to tweak it later on to work on suffixes or > > within > > >> phrases how should I go about that? > > >> > > >> Thanks again for your help. > > >> > > >> Regards, > > >> Salman > > >> > > >> > > >> On Sun, Dec 6, 2015 at 1:24 PM, Andrea Gazzarini < > a.gazzar...@gmail.com > > > > > >> wrote: > > >> > > >> > Hi Salman, > > >> > that's because you're using a StandardTokenizer. Try with something > > like > > >> > this (copied, pasted and changed using my phone so probably with a > lot > > >> of > > >> > mistakes ;) but you should be able to get what I mean). BTW I don't > > >> know if > > >> > that's the case but I would also put a MappingCharFilterFactory > > >> > > > >> > > >> > positionIncrementGap="100"> > > >> > > > >> > * > >> > mapping="mapping-FoldToASCII.txt"/>* > > >> > > > >> > > > >> > > >> > generateWordParts="0" generateNumberParts="0" catenateAll="1" > > >> > splitOnCaseChange="0" /> > > >> > > >> > maxGramSize="20"/> > > >> > > > >> > > > >> > * > >> > mapping="mapping-FoldToASCII.txt"/>* > > >> > > > >> > > > >> > > >> > generateWordParts="0" generateNumberParts="0" catenateAll="1" > > >> > splitOnCaseChange="0" /> > > >> > > > >> > > > >> > > > >> > > > >> > 2015-12-06 9:36 GMT+01:00 Salman Ansari : > > >> > > > >> > > Hi, > > >> > > > > >> > > > > >> > > > > >> > > I have updated my schema.xml as mentioned in the previous posts > > using > > >> > > > > >> > > > > >> > > > > >> > > > >> > > positionIncrementGap="100"> > > >> > > > > >> > > > > >> > > > > >> > > > >> minGramSize="1" > > >> > > maxGramSize="20"/> > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > This does the auto-complete, but it does it at every portion of > the > > >> text > > >> > > (not just at the beginning) (prefix). So searching for "And" in my > > >> field > > >> > > for locations returns both of the following documents. > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > 1 > > >> > > > > >> > > AD > > >> > > > > >> > > *And*orra > > >> > > > > >> > > أندورا > > >> > > > > >> > > 1519794717684924416 > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > 5 > > >> > > > > >> > > AG > > >> > > > > >> > > Antigua *and* Barbuda > > >> > > > > >> > > أنتيجوا وبربودا > > >> > > > > >> > > 1519794717701701633 > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > I have read about this and at first I thought I need to add > > >> side="front" > > >> > > but after adding that, Solr returned an error (when
Re: Solr Auto-Complete
Do you mean "phrase" or "term" prefixes? If you try to put a field value (two or more terms) in the analysis page you will see what the index analyzer chain (of my example field type) is doing. The whole value is managed as a single-ngrammed token, so you will get only a phrase prefix search, as in your request. If you want to manage also terms prefixes, I would also index another field (similar to the example you posted); then, the search handler with e(dismax) would have something like this: > text_suggestion_phrase_prefix_search^b1 text_suggestion_terms_prefix_search^b2 b1 and b2 values strictly depend on your search logic. Is that close that what you were looking for? Best, Andrea 2015-12-06 11:53 GMT+01:00 Salman Ansari: > Thanks a lot Andrea. It did work. > > However, just for my understanding, can you please explain more how did you > make it work for prefixes. I know you mentioned using another Tokenizer but > for example, if I want to tweak it later on to work on suffixes or within > phrases how should I go about that? > > Thanks again for your help. > > Regards, > Salman > > > On Sun, Dec 6, 2015 at 1:24 PM, Andrea Gazzarini > wrote: > > > Hi Salman, > > that's because you're using a StandardTokenizer. Try with something like > > this (copied, pasted and changed using my phone so probably with a lot of > > mistakes ;) but you should be able to get what I mean). BTW I don't know > if > > that's the case but I would also put a MappingCharFilterFactory > > > > > positionIncrementGap="100"> > > > > * > mapping="mapping-FoldToASCII.txt"/>* > > > > > > > generateWordParts="0" generateNumberParts="0" catenateAll="1" > > splitOnCaseChange="0" /> > > > maxGramSize="20"/> > > > > > > * > mapping="mapping-FoldToASCII.txt"/>* > > > > > > > generateWordParts="0" generateNumberParts="0" catenateAll="1" > > splitOnCaseChange="0" /> > > > > > > > > > > 2015-12-06 9:36 GMT+01:00 Salman Ansari : > > > > > Hi, > > > > > > > > > > > > I have updated my schema.xml as mentioned in the previous posts using > > > > > > > > > > > > > > positionIncrementGap="100"> > > > > > > > > > > > > > > maxGramSize="20"/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This does the auto-complete, but it does it at every portion of the > text > > > (not just at the beginning) (prefix). So searching for "And" in my > field > > > for locations returns both of the following documents. > > > > > > > > > > > > > > > > > > 1 > > > > > > AD > > > > > > *And*orra > > > > > > أندورا > > > > > > 1519794717684924416 > > > > > > > > > > > > > > > > > > 5 > > > > > > AG > > > > > > Antigua *and* Barbuda > > > > > > أنتيجوا وبربودا > > > > > > 1519794717701701633 > > > > > > > > > > > > > > > > > > I have read about this and at first I thought I need to add > side="front" > > > but after adding that, Solr returned an error (when creating a > > collection) > > > indicating "Unknown parameters
Re: Solr Auto-Complete
Hi, I have updated my schema.xml as mentioned in the previous posts using This does the auto-complete, but it does it at every portion of the text (not just at the beginning) (prefix). So searching for "And" in my field for locations returns both of the following documents. 1 AD *And*orra أندورا 1519794717684924416 5 AG Antigua *and* Barbuda أنتيجوا وبربودا 1519794717701701633 I have read about this and at first I thought I need to add side="front" but after adding that, Solr returned an error (when creating a collection) indicating "Unknown parameters
Re: Solr Auto-Complete
Hi Salman, that's because you're using a StandardTokenizer. Try with something like this (copied, pasted and changed using my phone so probably with a lot of mistakes ;) but you should be able to get what I mean). BTW I don't know if that's the case but I would also put a MappingCharFilterFactory ** ** 2015-12-06 9:36 GMT+01:00 Salman Ansari: > Hi, > > > > I have updated my schema.xml as mentioned in the previous posts using > > > > positionIncrementGap="100"> > > > > maxGramSize="20"/> > > > > > > > > > > This does the auto-complete, but it does it at every portion of the text > (not just at the beginning) (prefix). So searching for "And" in my field > for locations returns both of the following documents. > > > > > > 1 > > AD > > *And*orra > > أندورا > > 1519794717684924416 > > > > > > 5 > > AG > > Antigua *and* Barbuda > > أنتيجوا وبربودا > > 1519794717701701633 > > > > > > I have read about this and at first I thought I need to add side="front" > but after adding that, Solr returned an error (when creating a collection) > indicating "Unknown parameters
Re: Solr Auto-Complete
Thanks a lot Andrea. It did work. However, just for my understanding, can you please explain more how did you make it work for prefixes. I know you mentioned using another Tokenizer but for example, if I want to tweak it later on to work on suffixes or within phrases how should I go about that? Thanks again for your help. Regards, Salman On Sun, Dec 6, 2015 at 1:24 PM, Andrea Gazzariniwrote: > Hi Salman, > that's because you're using a StandardTokenizer. Try with something like > this (copied, pasted and changed using my phone so probably with a lot of > mistakes ;) but you should be able to get what I mean). BTW I don't know if > that's the case but I would also put a MappingCharFilterFactory > > positionIncrementGap="100"> > > * mapping="mapping-FoldToASCII.txt"/>* > > > generateWordParts="0" generateNumberParts="0" catenateAll="1" > splitOnCaseChange="0" /> > maxGramSize="20"/> > > > * mapping="mapping-FoldToASCII.txt"/>* > > > generateWordParts="0" generateNumberParts="0" catenateAll="1" > splitOnCaseChange="0" /> > > > > > 2015-12-06 9:36 GMT+01:00 Salman Ansari : > > > Hi, > > > > > > > > I have updated my schema.xml as mentioned in the previous posts using > > > > > > > > > positionIncrementGap="100"> > > > > > > > > > maxGramSize="20"/> > > > > > > > > > > > > > > > > > > > > This does the auto-complete, but it does it at every portion of the text > > (not just at the beginning) (prefix). So searching for "And" in my field > > for locations returns both of the following documents. > > > > > > > > > > > > 1 > > > > AD > > > > *And*orra > > > > أندورا > > > > 1519794717684924416 > > > > > > > > > > > > 5 > > > > AG > > > > Antigua *and* Barbuda > > > > أنتيجوا وبربودا > > > > 1519794717701701633 > > > > > > > > > > > > I have read about this and at first I thought I need to add side="front" > > but after adding that, Solr returned an error (when creating a > collection) > > indicating "Unknown parameters
Re: Solr Auto-Complete
Sorry, my damned mobile: "Is that close to what you were looking for?" 2015-12-06 12:07 GMT+01:00 Andrea Gazzarini: > Do you mean "phrase" or "term" prefixes? If you try to put a field value > (two or more terms) in the analysis page you will see what the index > analyzer chain (of my example field type) is doing. The whole value is > managed as a single-ngrammed token, so you will get only a phrase prefix > search, as in your request. > > If you want to manage also terms prefixes, I would also index another > field (similar to the example you posted); then, the search handler with > e(dismax) would have something like this: > > >> > text_suggestion_phrase_prefix_search^b1 > text_suggestion_terms_prefix_search^b2 > > > > > b1 and b2 values strictly depend on your search logic. > > Is that close that what you were looking for? > > Best, > Andrea > > > > 2015-12-06 11:53 GMT+01:00 Salman Ansari : > >> Thanks a lot Andrea. It did work. >> >> However, just for my understanding, can you please explain more how did >> you >> make it work for prefixes. I know you mentioned using another Tokenizer >> but >> for example, if I want to tweak it later on to work on suffixes or within >> phrases how should I go about that? >> >> Thanks again for your help. >> >> Regards, >> Salman >> >> >> On Sun, Dec 6, 2015 at 1:24 PM, Andrea Gazzarini >> wrote: >> >> > Hi Salman, >> > that's because you're using a StandardTokenizer. Try with something like >> > this (copied, pasted and changed using my phone so probably with a lot >> of >> > mistakes ;) but you should be able to get what I mean). BTW I don't >> know if >> > that's the case but I would also put a MappingCharFilterFactory >> > >> > > > positionIncrementGap="100"> >> > >> > *> > mapping="mapping-FoldToASCII.txt"/>* >> > >> > >> > > > generateWordParts="0" generateNumberParts="0" catenateAll="1" >> > splitOnCaseChange="0" /> >> > > > maxGramSize="20"/> >> > >> > >> > *> > mapping="mapping-FoldToASCII.txt"/>* >> > >> > >> > > > generateWordParts="0" generateNumberParts="0" catenateAll="1" >> > splitOnCaseChange="0" /> >> > >> > >> > >> > >> > 2015-12-06 9:36 GMT+01:00 Salman Ansari : >> > >> > > Hi, >> > > >> > > >> > > >> > > I have updated my schema.xml as mentioned in the previous posts using >> > > >> > > >> > > >> > > > > > positionIncrementGap="100"> >> > > >> > > >> > > >> > > > minGramSize="1" >> > > maxGramSize="20"/> >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > This does the auto-complete, but it does it at every portion of the >> text >> > > (not just at the beginning) (prefix). So searching for "And" in my >> field >> > > for locations returns both of the following documents. >> > > >> > > >> > > >> > > >> > > >> > > 1 >> > > >> > > AD >> > > >> > > *And*orra >> > > >> > > أندورا >> > > >> > > 1519794717684924416 >> > > >> > > >> > > >> > > >> > > >> > > 5 >> > > >> > > AG >> > > >> > > Antigua *and* Barbuda >> > > >> > > أنتيجوا وبربودا >> > > >> > > 1519794717701701633 >> > > >> > > >> > > >> > > >> > > >> > > I have read about this and at first I thought I need to add >> side="front" >> > > but after adding that, Solr returned an error (when creating a >> > collection) >> > > indicating "Unknown parameters
Re: Solr Auto-Complete
That is right. I am actually looking for phrase prefixes not each term prefix within the phrase. That satisfies my requirements. However, my additional question was how do I manipulate the filedType to later allow for suffix matches as well? or will that be a completely different fieldType definition? Regards, Salman On Sun, Dec 6, 2015 at 2:12 PM, Andrea Gazzariniwrote: > Sorry, my damned mobile: "Is that close to what you were looking for?" > > 2015-12-06 12:07 GMT+01:00 Andrea Gazzarini : > > > Do you mean "phrase" or "term" prefixes? If you try to put a field value > > (two or more terms) in the analysis page you will see what the index > > analyzer chain (of my example field type) is doing. The whole value is > > managed as a single-ngrammed token, so you will get only a phrase prefix > > search, as in your request. > > > > If you want to manage also terms prefixes, I would also index another > > field (similar to the example you posted); then, the search handler with > > e(dismax) would have something like this: > > > > > >> > > text_suggestion_phrase_prefix_search^b1 > > text_suggestion_terms_prefix_search^b2 > > > > > > > > > > b1 and b2 values strictly depend on your search logic. > > > > Is that close that what you were looking for? > > > > Best, > > Andrea > > > > > > > > 2015-12-06 11:53 GMT+01:00 Salman Ansari : > > > >> Thanks a lot Andrea. It did work. > >> > >> However, just for my understanding, can you please explain more how did > >> you > >> make it work for prefixes. I know you mentioned using another Tokenizer > >> but > >> for example, if I want to tweak it later on to work on suffixes or > within > >> phrases how should I go about that? > >> > >> Thanks again for your help. > >> > >> Regards, > >> Salman > >> > >> > >> On Sun, Dec 6, 2015 at 1:24 PM, Andrea Gazzarini > > >> wrote: > >> > >> > Hi Salman, > >> > that's because you're using a StandardTokenizer. Try with something > like > >> > this (copied, pasted and changed using my phone so probably with a lot > >> of > >> > mistakes ;) but you should be able to get what I mean). BTW I don't > >> know if > >> > that's the case but I would also put a MappingCharFilterFactory > >> > > >> > >> > positionIncrementGap="100"> > >> > > >> > * >> > mapping="mapping-FoldToASCII.txt"/>* > >> > > >> > > >> > >> > generateWordParts="0" generateNumberParts="0" catenateAll="1" > >> > splitOnCaseChange="0" /> > >> > >> > maxGramSize="20"/> > >> > > >> > > >> > * >> > mapping="mapping-FoldToASCII.txt"/>* > >> > > >> > > >> > >> > generateWordParts="0" generateNumberParts="0" catenateAll="1" > >> > splitOnCaseChange="0" /> > >> > > >> > > >> > > >> > > >> > 2015-12-06 9:36 GMT+01:00 Salman Ansari : > >> > > >> > > Hi, > >> > > > >> > > > >> > > > >> > > I have updated my schema.xml as mentioned in the previous posts > using > >> > > > >> > > > >> > > > >> > > >> > > positionIncrementGap="100"> > >> > > > >> > > > >> > > > >> > > >> minGramSize="1" > >> > > maxGramSize="20"/> > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > This does the auto-complete, but it does it at every portion of the > >> text > >> > > (not just at the beginning) (prefix). So searching for "And" in my > >> field > >> > > for locations returns both of the following documents. > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > 1 > >> > > > >> > > AD > >> > > > >> > > *And*orra > >> > > > >> > > أندورا > >> > > > >> > > 1519794717684924416 > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > 5 > >> > > > >> > > AG > >> > > > >> > > Antigua *and* Barbuda > >> > > > >> > > أنتيجوا وبربودا > >> > > > >> > > 1519794717701701633 > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > I have read about this and at first I thought I need to add > >> side="front" > >> > > but after adding that, Solr returned an error (when creating a > >> > collection) > >> > > indicating "Unknown parameters
Re: Solr Auto-Complete
You can see an example of similar use at: http://www.solr-start.com/javadoc/solr-lucene/index.html (search box). The corresponding schema is here: https://github.com/arafalov/Solr-Javadoc/blob/master/JavadocIndex/JavadocCollection/conf/schema.xml#L24 . It does have some extra special-case stuff to allow to search by the fragments, but the general use case is the same. Regards, Alex. Newsletter and resources for Solr beginners and intermediates: http://www.solr-start.com/ On 4 December 2015 at 10:11, Salman Ansariwrote: > Thanks Alan, Alessandaro and Andrea for your great explanations. I will > follow the path of adding edge ngrams to the field type for my use case. > > Regards, > Salman > > On Thu, Dec 3, 2015 at 12:23 PM, Alessandro Benedetti > wrote: > >> "Sounds good but I heard "/suggest" component is the recommended way of >> doing auto-complete" >> >> This sounds fantastic :) >> We "heard" that as well, we know what the suggest component does. >> The point is that you would like to retrieve the suggestions + some >> consistent payload in different fields. >> Current suggest component offers some effort in providing a payload, but >> almost all the suggester implementation are based on an FST approach which >> aim to be as fast and memory efficient as possible. >> Honestly you could experiment and even contribute a customisation if you >> want to add a new feature to the suggest component able to return complex >> payloads together with the suggestions. >> Apart that, it strictly depends of how you want to provide the >> autocompletion, there are plenty of different lookups implementation and >> plenty of tokenizer/token filters to combine . >> So I would confirm what we already said and that Andrea confirmed. >> >> If anyone has played with the suggester suggestions payload, his feedback >> is welcome! >> >> Cheers >> >> >> On 3 December 2015 at 06:21, Andrea Gazzarini >> wrote: >> >> > Hi Salman, >> > few months ago I have been involved in a project similar to >> > map.geoadmin.ch >> > and there, I had your same need (I also sent an email to this list). >> > >> > From my side I can furtherly confirm what Alan and Alessandro already >> > explained, I followed that approach. >> > >> > IMHO, that is the "recommended way" if the component's features meet your >> > needs (i.e. do not reinvent the wheel) but it seems you're out of those >> > bounds. >> > >> > Best, >> > Andrea >> > On 2 Dec 2015 21:51, "Salman Ansari" wrote: >> > >> > > Sounds good but I heard "/suggest" component is the recommended way of >> > > doing auto-complete in the new versions of Solr. Something along the >> > lines >> > > of this article >> > > https://cwiki.apache.org/confluence/display/solr/Suggester >> > > >> > > >> > > >> > > mySuggester >> > > FuzzyLookupFactory >> > > DocumentDictionaryFactory >> > > cat >> > > price >> > > string >> > > false >> > > >> > > >> > > >> > > Can someone confirm this? >> > > >> > > Regards, >> > > Salman >> > > >> > > >> > > On Wed, Dec 2, 2015 at 1:14 PM, Alessandro Benedetti < >> > > abenede...@apache.org> >> > > wrote: >> > > >> > > > Hi Salman, >> > > > I agree with Alan. >> > > > Just configure your schema with the proper analysers . >> > > > For the field you want to use for suggestions you are likely to need >> > > simply >> > > > this fieldType : >> > > > >> > > > > > > > positionIncrementGap="100"> >> > > > >> > > > >> > > > >> > > > > minGramSize="1" >> > > > maxGramSize="20"/> >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > This is a very sample example, please adapt it to your use case. >> > > > >> > > > Cheers >> > > > >> > > > On 2 December 2015 at 09:41, Alan Woodward wrote: >> > > > >> > > > > Hi Salman, >> > > > > >> > > > > It sounds as though you want to do a normal search against a >> special >> > > > > 'suggest' field, that's been indexed with edge ngrams. >> > > > > >> > > > > Alan Woodward >> > > > > www.flax.co.uk >> > > > > >> > > > > >> > > > > On 2 Dec 2015, at 09:31, Salman Ansari wrote: >> > > > > >> > > > > > Hi, >> > > > > > >> > > > > > I am looking for auto-complete in Solr but on top of just auto >> > > > complete I >> > > > > > want as well to return the data completely (not just >> suggestions), >> > > so I >> > > > > > want to get back the ids, and other fields in the whole >> document. I >> > > > tried >> > > > > > the following 2 approaches but each had issues >> > > > > > >> > > > > > 1) Used the /suggest component but that returns a very specific >> > > format >> > > > > > which looks like I cannot customize. I want to return the whole >> > > > document >> > > > > > that has a matching field and not only the suggestion list. So >> for >> > > > > example, >> > > >
Re: Solr Auto-Complete
Thanks Alan, Alessandaro and Andrea for your great explanations. I will follow the path of adding edge ngrams to the field type for my use case. Regards, Salman On Thu, Dec 3, 2015 at 12:23 PM, Alessandro Benedettiwrote: > "Sounds good but I heard "/suggest" component is the recommended way of > doing auto-complete" > > This sounds fantastic :) > We "heard" that as well, we know what the suggest component does. > The point is that you would like to retrieve the suggestions + some > consistent payload in different fields. > Current suggest component offers some effort in providing a payload, but > almost all the suggester implementation are based on an FST approach which > aim to be as fast and memory efficient as possible. > Honestly you could experiment and even contribute a customisation if you > want to add a new feature to the suggest component able to return complex > payloads together with the suggestions. > Apart that, it strictly depends of how you want to provide the > autocompletion, there are plenty of different lookups implementation and > plenty of tokenizer/token filters to combine . > So I would confirm what we already said and that Andrea confirmed. > > If anyone has played with the suggester suggestions payload, his feedback > is welcome! > > Cheers > > > On 3 December 2015 at 06:21, Andrea Gazzarini > wrote: > > > Hi Salman, > > few months ago I have been involved in a project similar to > > map.geoadmin.ch > > and there, I had your same need (I also sent an email to this list). > > > > From my side I can furtherly confirm what Alan and Alessandro already > > explained, I followed that approach. > > > > IMHO, that is the "recommended way" if the component's features meet your > > needs (i.e. do not reinvent the wheel) but it seems you're out of those > > bounds. > > > > Best, > > Andrea > > On 2 Dec 2015 21:51, "Salman Ansari" wrote: > > > > > Sounds good but I heard "/suggest" component is the recommended way of > > > doing auto-complete in the new versions of Solr. Something along the > > lines > > > of this article > > > https://cwiki.apache.org/confluence/display/solr/Suggester > > > > > > > > > > > > mySuggester > > > FuzzyLookupFactory > > > DocumentDictionaryFactory > > > cat > > > price > > > string > > > false > > > > > > > > > > > > Can someone confirm this? > > > > > > Regards, > > > Salman > > > > > > > > > On Wed, Dec 2, 2015 at 1:14 PM, Alessandro Benedetti < > > > abenede...@apache.org> > > > wrote: > > > > > > > Hi Salman, > > > > I agree with Alan. > > > > Just configure your schema with the proper analysers . > > > > For the field you want to use for suggestions you are likely to need > > > simply > > > > this fieldType : > > > > > > > > > > > positionIncrementGap="100"> > > > > > > > > > > > > > > > > minGramSize="1" > > > > maxGramSize="20"/> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is a very sample example, please adapt it to your use case. > > > > > > > > Cheers > > > > > > > > On 2 December 2015 at 09:41, Alan Woodward wrote: > > > > > > > > > Hi Salman, > > > > > > > > > > It sounds as though you want to do a normal search against a > special > > > > > 'suggest' field, that's been indexed with edge ngrams. > > > > > > > > > > Alan Woodward > > > > > www.flax.co.uk > > > > > > > > > > > > > > > On 2 Dec 2015, at 09:31, Salman Ansari wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I am looking for auto-complete in Solr but on top of just auto > > > > complete I > > > > > > want as well to return the data completely (not just > suggestions), > > > so I > > > > > > want to get back the ids, and other fields in the whole > document. I > > > > tried > > > > > > the following 2 approaches but each had issues > > > > > > > > > > > > 1) Used the /suggest component but that returns a very specific > > > format > > > > > > which looks like I cannot customize. I want to return the whole > > > > document > > > > > > that has a matching field and not only the suggestion list. So > for > > > > > example, > > > > > > if I write "hard" it returns the results in a specific format as > > > > follows > > > > > > > > > > > > hard drive > > > > > > hard disk > > > > > > > > > > > > Is there a way to get back additional fields with suggestions? > > > > > > > > > > > > 2) Tried the normal /select component but that does not do > > > > auto-complete > > > > > on > > > > > > portion of the word. So, for example, if I write the query as > > "bara" > > > it > > > > > > DOES NOT return "barack obama". Any suggestions how to solve > this? > > > > > > > > > > > > > > > > > > Regards, > > > > > > Salman > > > > > > > > > > > > > > > > > > > > > > -- > > > > -- > > > > > > > >
Re: Solr Auto-Complete
"Sounds good but I heard "/suggest" component is the recommended way of doing auto-complete" This sounds fantastic :) We "heard" that as well, we know what the suggest component does. The point is that you would like to retrieve the suggestions + some consistent payload in different fields. Current suggest component offers some effort in providing a payload, but almost all the suggester implementation are based on an FST approach which aim to be as fast and memory efficient as possible. Honestly you could experiment and even contribute a customisation if you want to add a new feature to the suggest component able to return complex payloads together with the suggestions. Apart that, it strictly depends of how you want to provide the autocompletion, there are plenty of different lookups implementation and plenty of tokenizer/token filters to combine . So I would confirm what we already said and that Andrea confirmed. If anyone has played with the suggester suggestions payload, his feedback is welcome! Cheers On 3 December 2015 at 06:21, Andrea Gazzariniwrote: > Hi Salman, > few months ago I have been involved in a project similar to > map.geoadmin.ch > and there, I had your same need (I also sent an email to this list). > > From my side I can furtherly confirm what Alan and Alessandro already > explained, I followed that approach. > > IMHO, that is the "recommended way" if the component's features meet your > needs (i.e. do not reinvent the wheel) but it seems you're out of those > bounds. > > Best, > Andrea > On 2 Dec 2015 21:51, "Salman Ansari" wrote: > > > Sounds good but I heard "/suggest" component is the recommended way of > > doing auto-complete in the new versions of Solr. Something along the > lines > > of this article > > https://cwiki.apache.org/confluence/display/solr/Suggester > > > > > > > > mySuggester > > FuzzyLookupFactory > > DocumentDictionaryFactory > > cat > > price > > string > > false > > > > > > > > Can someone confirm this? > > > > Regards, > > Salman > > > > > > On Wed, Dec 2, 2015 at 1:14 PM, Alessandro Benedetti < > > abenede...@apache.org> > > wrote: > > > > > Hi Salman, > > > I agree with Alan. > > > Just configure your schema with the proper analysers . > > > For the field you want to use for suggestions you are likely to need > > simply > > > this fieldType : > > > > > > > > positionIncrementGap="100"> > > > > > > > > > > > > > > maxGramSize="20"/> > > > > > > > > > > > > > > > > > > > > > > > > This is a very sample example, please adapt it to your use case. > > > > > > Cheers > > > > > > On 2 December 2015 at 09:41, Alan Woodward wrote: > > > > > > > Hi Salman, > > > > > > > > It sounds as though you want to do a normal search against a special > > > > 'suggest' field, that's been indexed with edge ngrams. > > > > > > > > Alan Woodward > > > > www.flax.co.uk > > > > > > > > > > > > On 2 Dec 2015, at 09:31, Salman Ansari wrote: > > > > > > > > > Hi, > > > > > > > > > > I am looking for auto-complete in Solr but on top of just auto > > > complete I > > > > > want as well to return the data completely (not just suggestions), > > so I > > > > > want to get back the ids, and other fields in the whole document. I > > > tried > > > > > the following 2 approaches but each had issues > > > > > > > > > > 1) Used the /suggest component but that returns a very specific > > format > > > > > which looks like I cannot customize. I want to return the whole > > > document > > > > > that has a matching field and not only the suggestion list. So for > > > > example, > > > > > if I write "hard" it returns the results in a specific format as > > > follows > > > > > > > > > > hard drive > > > > > hard disk > > > > > > > > > > Is there a way to get back additional fields with suggestions? > > > > > > > > > > 2) Tried the normal /select component but that does not do > > > auto-complete > > > > on > > > > > portion of the word. So, for example, if I write the query as > "bara" > > it > > > > > DOES NOT return "barack obama". Any suggestions how to solve this? > > > > > > > > > > > > > > > Regards, > > > > > Salman > > > > > > > > > > > > > > > > > -- > > > -- > > > > > > Benedetti Alessandro > > > Visiting card : http://about.me/alessandro_benedetti > > > > > > "Tyger, tyger burning bright > > > In the forests of the night, > > > What immortal hand or eye > > > Could frame thy fearful symmetry?" > > > > > > William Blake - Songs of Experience -1794 England > > > > > > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: Solr Auto-Complete
Sounds good but I heard "/suggest" component is the recommended way of doing auto-complete in the new versions of Solr. Something along the lines of this article https://cwiki.apache.org/confluence/display/solr/Suggester mySuggester FuzzyLookupFactory DocumentDictionaryFactory cat price string false Can someone confirm this? Regards, Salman On Wed, Dec 2, 2015 at 1:14 PM, Alessandro Benedettiwrote: > Hi Salman, > I agree with Alan. > Just configure your schema with the proper analysers . > For the field you want to use for suggestions you are likely to need simply > this fieldType : > > positionIncrementGap="100"> > > > > maxGramSize="20"/> > > > > > > > > This is a very sample example, please adapt it to your use case. > > Cheers > > On 2 December 2015 at 09:41, Alan Woodward wrote: > > > Hi Salman, > > > > It sounds as though you want to do a normal search against a special > > 'suggest' field, that's been indexed with edge ngrams. > > > > Alan Woodward > > www.flax.co.uk > > > > > > On 2 Dec 2015, at 09:31, Salman Ansari wrote: > > > > > Hi, > > > > > > I am looking for auto-complete in Solr but on top of just auto > complete I > > > want as well to return the data completely (not just suggestions), so I > > > want to get back the ids, and other fields in the whole document. I > tried > > > the following 2 approaches but each had issues > > > > > > 1) Used the /suggest component but that returns a very specific format > > > which looks like I cannot customize. I want to return the whole > document > > > that has a matching field and not only the suggestion list. So for > > example, > > > if I write "hard" it returns the results in a specific format as > follows > > > > > > hard drive > > > hard disk > > > > > > Is there a way to get back additional fields with suggestions? > > > > > > 2) Tried the normal /select component but that does not do > auto-complete > > on > > > portion of the word. So, for example, if I write the query as "bara" it > > > DOES NOT return "barack obama". Any suggestions how to solve this? > > > > > > > > > Regards, > > > Salman > > > > > > > -- > -- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England >
Re: Solr Auto-Complete
Hi Salman, few months ago I have been involved in a project similar to map.geoadmin.ch and there, I had your same need (I also sent an email to this list). >From my side I can furtherly confirm what Alan and Alessandro already explained, I followed that approach. IMHO, that is the "recommended way" if the component's features meet your needs (i.e. do not reinvent the wheel) but it seems you're out of those bounds. Best, Andrea On 2 Dec 2015 21:51, "Salman Ansari"wrote: > Sounds good but I heard "/suggest" component is the recommended way of > doing auto-complete in the new versions of Solr. Something along the lines > of this article > https://cwiki.apache.org/confluence/display/solr/Suggester > > > > mySuggester > FuzzyLookupFactory > DocumentDictionaryFactory > cat > price > string > false > > > > Can someone confirm this? > > Regards, > Salman > > > On Wed, Dec 2, 2015 at 1:14 PM, Alessandro Benedetti < > abenede...@apache.org> > wrote: > > > Hi Salman, > > I agree with Alan. > > Just configure your schema with the proper analysers . > > For the field you want to use for suggestions you are likely to need > simply > > this fieldType : > > > > > positionIncrementGap="100"> > > > > > > > > > maxGramSize="20"/> > > > > > > > > > > > > > > > > This is a very sample example, please adapt it to your use case. > > > > Cheers > > > > On 2 December 2015 at 09:41, Alan Woodward wrote: > > > > > Hi Salman, > > > > > > It sounds as though you want to do a normal search against a special > > > 'suggest' field, that's been indexed with edge ngrams. > > > > > > Alan Woodward > > > www.flax.co.uk > > > > > > > > > On 2 Dec 2015, at 09:31, Salman Ansari wrote: > > > > > > > Hi, > > > > > > > > I am looking for auto-complete in Solr but on top of just auto > > complete I > > > > want as well to return the data completely (not just suggestions), > so I > > > > want to get back the ids, and other fields in the whole document. I > > tried > > > > the following 2 approaches but each had issues > > > > > > > > 1) Used the /suggest component but that returns a very specific > format > > > > which looks like I cannot customize. I want to return the whole > > document > > > > that has a matching field and not only the suggestion list. So for > > > example, > > > > if I write "hard" it returns the results in a specific format as > > follows > > > > > > > > hard drive > > > > hard disk > > > > > > > > Is there a way to get back additional fields with suggestions? > > > > > > > > 2) Tried the normal /select component but that does not do > > auto-complete > > > on > > > > portion of the word. So, for example, if I write the query as "bara" > it > > > > DOES NOT return "barack obama". Any suggestions how to solve this? > > > > > > > > > > > > Regards, > > > > Salman > > > > > > > > > > > > -- > > -- > > > > Benedetti Alessandro > > Visiting card : http://about.me/alessandro_benedetti > > > > "Tyger, tyger burning bright > > In the forests of the night, > > What immortal hand or eye > > Could frame thy fearful symmetry?" > > > > William Blake - Songs of Experience -1794 England > > >
Solr Auto-Complete
Hi, I am looking for auto-complete in Solr but on top of just auto complete I want as well to return the data completely (not just suggestions), so I want to get back the ids, and other fields in the whole document. I tried the following 2 approaches but each had issues 1) Used the /suggest component but that returns a very specific format which looks like I cannot customize. I want to return the whole document that has a matching field and not only the suggestion list. So for example, if I write "hard" it returns the results in a specific format as follows hard drive hard disk Is there a way to get back additional fields with suggestions? 2) Tried the normal /select component but that does not do auto-complete on portion of the word. So, for example, if I write the query as "bara" it DOES NOT return "barack obama". Any suggestions how to solve this? Regards, Salman
Re: Solr Auto-Complete
Hi Salman, It sounds as though you want to do a normal search against a special 'suggest' field, that's been indexed with edge ngrams. Alan Woodward www.flax.co.uk On 2 Dec 2015, at 09:31, Salman Ansari wrote: > Hi, > > I am looking for auto-complete in Solr but on top of just auto complete I > want as well to return the data completely (not just suggestions), so I > want to get back the ids, and other fields in the whole document. I tried > the following 2 approaches but each had issues > > 1) Used the /suggest component but that returns a very specific format > which looks like I cannot customize. I want to return the whole document > that has a matching field and not only the suggestion list. So for example, > if I write "hard" it returns the results in a specific format as follows > > hard drive > hard disk > > Is there a way to get back additional fields with suggestions? > > 2) Tried the normal /select component but that does not do auto-complete on > portion of the word. So, for example, if I write the query as "bara" it > DOES NOT return "barack obama". Any suggestions how to solve this? > > > Regards, > Salman
Re: Solr Auto-Complete
Hi Salman, I agree with Alan. Just configure your schema with the proper analysers . For the field you want to use for suggestions you are likely to need simply this fieldType : This is a very sample example, please adapt it to your use case. Cheers On 2 December 2015 at 09:41, Alan Woodwardwrote: > Hi Salman, > > It sounds as though you want to do a normal search against a special > 'suggest' field, that's been indexed with edge ngrams. > > Alan Woodward > www.flax.co.uk > > > On 2 Dec 2015, at 09:31, Salman Ansari wrote: > > > Hi, > > > > I am looking for auto-complete in Solr but on top of just auto complete I > > want as well to return the data completely (not just suggestions), so I > > want to get back the ids, and other fields in the whole document. I tried > > the following 2 approaches but each had issues > > > > 1) Used the /suggest component but that returns a very specific format > > which looks like I cannot customize. I want to return the whole document > > that has a matching field and not only the suggestion list. So for > example, > > if I write "hard" it returns the results in a specific format as follows > > > > hard drive > > hard disk > > > > Is there a way to get back additional fields with suggestions? > > > > 2) Tried the normal /select component but that does not do auto-complete > on > > portion of the word. So, for example, if I write the query as "bara" it > > DOES NOT return "barack obama". Any suggestions how to solve this? > > > > > > Regards, > > Salman > > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England