Um sorry for any confusion. I meant to say I solved my issue by inserting a charFilter before the WhitespaceTokenizerFactory to convert my problem word to a searchable form. I had a cut n paste malfunction below. Thanks guys.
-----Original Message----- From: Robert Petersen [mailto:rober...@buy.com] Sent: Tuesday, July 19, 2011 11:06 AM To: solr-user@lucene.apache.org Subject: RE: Analysis page output vs. actually getting search matches, a discrepency? Thanks Eric, Unfortunately I'm stemming the same on both sides, similar to the SOLR example settings for the text type field. Default search field is moreWords, as I want yes. Since I don't have this problem for any other mfg names at all in our index of almost 10 mm product docs, and this shows that is should match in my best estimation. Note: LucidKStemFilterFactory does not take 'Sterling' down to 'Sterl' in indexing nor searching, it stays as 'Sterling'. I have given up on this. I've decided it is just an unexplainable anomaly, and have solved it by inserting a LucidKStemFilterFactory and just modifying that word to it's searchable form before hitting the WhitespaceTokenizerFactory, which is kind of hackish but solves my problem at least. This seller only has a couple hundred cheap products on our site, so I have bigger fish to fry at this point. I've wasted too much time trying to chase this down. Cheers all Robi -----Original Message----- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Monday, July 18, 2011 5:33 PM To: solr-user@lucene.apache.org Subject: Re: Analysis page output vs. actually getting search matches, a discrepency? Hmmm, is there any chance that you're stemming one place and not the other? And I infer from your output that your default search field is "moreWords", is that true and expected? You might use luke or the TermsComponent to see what's actually in the index, I'm going to guess that you'll find "sterl" but not "sterling" as an indexed term and your problem is stemming, but that's a shot in the dark..... Best Erick On Mon, Jul 18, 2011 at 5:37 PM, Robert Petersen <rober...@buy.com> wrote: > OK I did what Hoss said, it only confirms I don't get a match when I > should and that the query parser is doing the expected. Here are the > details for one test sku. > > My analysis page output is shown in my email starting this thread and > here is my query debug output. This absolutely should match but > doesn't. Both the indexing side and the query side are splitting on > case changes. This actually isn't a problem for any of our other > content, for instance there is no issue searching for 'VideoSecu'. > Their products come up fine in our searches regardless of casing in the > query. Only SterlingTek's products seem to be causing us issues. > > Indexed content has camel case, stored in the text field 'moreWords': > "SterlingTek's NB-2LH 2 Pack Batteries + Charger Combo for Canon DC301" > Search term not matching with camel case: "SterlingTek's" > Search term matching if no case changes: "Sterlingtek's" > > Indexing: > <filter class="solr.WordDelimiterFilterFactory" > generateWordParts="1" > generateNumberParts="1" > catenateWords="1" > catenateNumbers="1" > catenateAll="0" > splitOnCaseChange="1" > preserveOriginal="0" > /> > Searching: > <filter class="solr.WordDelimiterFilterFactory" > generateWordParts="1" > generateNumberParts="1" > catenateWords="0" > catenateNumbers="0" > catenateAll="0" > splitOnCaseChange="1" > preserveOriginal="0" > /> > > Thanks > > http://ssdevrh01.buy.com:8983/solr/10000/select?indent=on&version=2.2&q= > SterlingTek%27s&fq=&start=0&rows=1&fl=*%2Cscore&qt=standard&wt=standard& > debugQuery=on&explainOther=sku%3A216473417&hl=on&hl.fl=&echoHandler=true > &adf > > <response> > <lst name="responseHeader"> > <int name="status">0</int> > <int name="QTime">4</int> > <str > name="handler">org.apache.solr.handler.component.SearchHandler</str> > <lst name="params"> > <str name="explainOther">sku:216473417</str> > <str name="indent">on</str> > <str name="echoHandler">true</str> > <str name="hl.fl"/> > <str name="wt">standard</str> > <str name="hl">on</str> > <str name="rows">1</str> > <str name="version">2.2</str> > <str name="fl">*,score</str> > <str name="debugQuery">on</str> > <str name="start">0</str> > <str name="q">SterlingTek's</str> > <str name="qt">standard</str> > <str name="fq"/> > </lst> > </lst> > <result name="response" numFound="0" start="0" maxScore="0.0"/> > <lst name="highlighting"/> > <lst name="debug"> > <str name="rawquerystring">SterlingTek's</str> > <str name="querystring">SterlingTek's</str> > <str name="parsedquery">PhraseQuery(moreWords:"sterling tek")</str> > <str name="parsedquery_toString">moreWords:"sterling tek"</str> > <lst name="explain"/> > <str name="otherQuery">sku:216473417</str> > <lst name="explainOther"> > <str name="216473417"> > 0.0 = fieldWeight(moreWords:"sterling tek" in 76351), product of: > 0.0 = tf(phraseFreq=0.0) > 19.502613 = idf(moreWords: sterling=1 tek=72) > 0.15625 = fieldNorm(field=moreWords, doc=76351) > > </str> > </lst> > <str name="QParser">LuceneQParser > </str> > <arr name="filter_queries"> > <str/> > </arr> > > > > -----Original Message----- > From: Chris Hostetter [mailto:hossman_luc...@fucit.org] > Sent: Friday, July 15, 2011 4:36 PM > To: solr-user@lucene.apache.org > Subject: Re: Analysis page output vs. actually getting search matches, a > discrepency? > > > : Subject: Analysis page output vs. actually getting search matches, > : a discrepency? > > 99% of the time when people ask questions like this, it's because of > confusion about how/when QueryParsing comes into play (as opposed to > analysis) -- analysis.jsp only shows you part of the equation, it > doesn't > know what query parser you are using. > > you mentioned that you aren't getting matches when you expect them, and > you provided the analysis.jsp output, but you didn't mention anything > about the request you are making, the query parser used etc.... it > owuld > be good to know the full query URL, along with the debugQuery output > showing the final query toString info. > > if that info doesn't clear up the discrepency, you should also take a > look > at the explainOther info for the doc that you expect to match that isn't > > -- if you still aren't sure what's going on, post all of that info to > solr-user and folks can probably help you make sense of it. > > (all that said: in some instances this type of problem is simply that > someone changed the schema and didn't reindex everything, so the indexed > > terms don't really match what you think they do) > > > -Hoss >
