Re: Slow autocomplete(terms)
Thanks for helping me so far, Yes i have seen the edgeNGrams possiblity. Correct me if i'm wrong, but i thought it isn't possible to do infix searches with edgeNGrams? Like chest gives suggestion manchester. -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-performance-problem-tp3351352p3361155.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow autocomplete(terms)
Roy, Use something other than Nabble or quote previous email to help people keep track of what your problem is/was about. Yes, with edge ngrams you won't be able to do infix searches but are you sure you want that? People typically don't miss/skip the beginning of a word... Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message - From: roySolr royrutten1...@gmail.com To: solr-user@lucene.apache.org Cc: Sent: Friday, September 23, 2011 3:15 AM Subject: Re: Slow autocomplete(terms) T hanks for helping me so far, Yes i have seen the edgeNGrams possiblity. Correct me if i'm wrong, but i thought it isn't possible to do infix searches with edgeNGrams? Like chest gives suggestion manchester. -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-performance-problem-tp3351352p3361155.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow autocomplete(terms)
yes it is possible http://www.medihack.org/2011/03/01/autocompletion-autosuggestion-using-solr/ Since i m looking into autosuggest i came across that info while doing research.. -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-performance-problem-tp3351352p3362071.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow autocomplete(terms)
Hello Erick, Thanks for your answer but i have some problems with the ngramfilter. My conf look like this: fieldType name=autocomplete class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.NGramFilterFactory minGramSize=2 maxGramSize=8/ /analyzer analyzer type=query tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.TrimFilterFactory/ /analyzer /fieldType I see this in the analysis: manchester ma an nc ch he es st te er man anc nch che hes est ste ter mancanchnche cheshestestestermanch anche nches chest heste ester manche anches nchest cheste hester manches anchest ncheste chester manchestancheste nchester When i use terms i see all this results back in the response. So i type ches i got this: ches nches anches nchest ncheste I want one suggestion with a total keyword: manchester. Is this possible? -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-performance-problem-tp3351352p3358126.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow autocomplete(terms)
not sure if u already seen this but may be useful http://www.lucidimagination.com/blog/2009/09/08/auto-suggest-from-popular-queries-using-edgengrams/ -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-performance-problem-tp3351352p3360663.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow autocomplete(terms)
Hi Roy, Try edge ngram instead. See also: http://sematext.com/products/autocomplete/index.html (comes with a nice UI, a bunch of configurable things, etc.) Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message - From: roySolr royrutten1...@gmail.com To: solr-user@lucene.apache.org Cc: Sent: Thursday, September 22, 2011 5:02 AM Subject: Re: Slow autocomplete(terms) Hello Erick, Thanks for your answer but i have some problems with the ngramfilter. My conf look like this: fieldType name=autocomplete class=solr.TextField positionIncrementGap=100 analyzer type=index tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.NGramFilterFactory minGramSize=2 maxGramSize=8/ /analyzer analyzer type=query tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.TrimFilterFactory/ /analyzer /fieldType I see this in the analysis: manchester ma an nc ch he es st te er man anc nch che hes est ste ter manc anch nche ches hest este ster manch anche nches chest heste ester manche anches nchest cheste hester manches anchest ncheste chester manchest ancheste nchester When i use terms i see all this results back in the response. So i type ches i got this: ches nches anches nchest ncheste I want one suggestion with a total keyword: manchester. Is this possible? -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-performance-problem-tp3351352p3358126.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Slow autocomplete(terms)
Think about ngrams if you really need infix searches, you're right that the regex is very probably the root of your problem. The index has to examine *every* term in the field to determine if the regex will match. Best Erick On Tue, Sep 20, 2011 at 12:57 AM, roySolr royrutten1...@gmail.com wrote: Hello, I used the terms request for autocomplete. It works fine with 200.000 records but with 2 million docs it's very slow.. I use some regex to fix autocomplete in the middle of words, example: chest - manchester. My call(pecl PHP solr): $query = new SolrQuery(); $query-setTermsLimit(10); $query-setTerms(true); $query-setTermsField($field); $term = SolrUtils::escapeQueryChars ($term); $query-set(terms.regex,(.*)$term(.*)); $query-set(terms.regex.flag,case_insensitive); URL: /solr/terms?terms.fl=autocompletewhatterms.regex=(.*)chest(.*)terms.regex.flag=case_insensitiveterms=true I think the regex is the reason for the very high query time: Solr search between 2 million docs with a regex. The query takes 2 seconds, this is to much for the autocomplete. A user typed manchester united and solr needs to do 16 query's from 2 seconds. Are there some other options? Faster solutions? I use solr 3.1 -- View this message in context: http://lucene.472066.n3.nabble.com/Slow-autocomplete-terms-tp3351352p3351352.html Sent from the Solr - User mailing list archive at Nabble.com.
Slow autocomplete(terms)
Hello, I used the terms request for autocomplete. It works fine with 200.000 records but with 2 million docs it's very slow.. I use some regex to fix autocomplete in the middle of words, example: chest - manchester. My call(pecl PHP solr): $query = new SolrQuery(); $query-setTermsLimit(10); $query-setTerms(true); $query-setTermsField($field); $term = SolrUtils::escapeQueryChars ($term); $query-set(terms.regex,(.*)$term(.*)); $query-set(terms.regex.flag,case_insensitive); URL: /solr/terms?terms.fl=autocompletewhatterms.regex=(.*)chest(.*)terms.regex.flag=case_insensitiveterms=true I think the regex is the reason for the very high query time: Solr search between 2 million docs with a regex. The query takes 2 seconds, this is to much for the autocomplete. A user typed manchester united and solr needs to do 16 query's from 2 seconds. Are there some other options? Faster solutions? I use solr 3.1 -- View this message in context: http://lucene.472066.n3.nabble.com/Slow-autocomplete-terms-tp3351352p3351352.html Sent from the Solr - User mailing list archive at Nabble.com.
