Re: StopFilterFactory and qf containing some fields that use it and some that do not

2012-02-01 Thread Jan Høydahl
Reviving this thread. You say: I do wonder...what if (e)dismax had a flag you could set that would tell it that if any analyzers removed a term, then that term would become optional for any fields for which it remained? I'm not sure what the development effort would perhaps it would be a

Re: StopFilterFactory and qf containing some fields that use it and some that do not

2011-01-13 Thread Jonathan Rochkind
It's a known 'issue' in dismax, (really an inherent part of dismax's design with no clear way to do anything about it), that qf over fields with different stop word definitions will produce odd results for a query with a stopword. Here's my understanding of what's going on:

RE: StopFilterFactory and qf containing some fields that use it and some that do not

2011-01-13 Thread Dyer, James
I appreciate the reply and blog posting. For now, I just enabled stopwords for all the fields on Qf. We have a very short list anyhow and our legacy search engine didn't even allow field-by-field configuration (stopwords are global on that system). I do wonder...what if (e)dismax had a flag

StopFilterFactory and qf containing some fields that use it and some that do not

2011-01-12 Thread Dyer, James
I'm running into a problem with StopFilterFactory in conjunction with (e)dismax queries that have a mix of fields, only some of which use StopFilterFactory. It seems that if even 1 field on the qf parameter does not use StopFilterFactory, then stop words are not removed when searching any

Re: StopFilterFactory and qf containing some fields that use it and some that do not

2011-01-12 Thread Markus Jelsma
I haven't used edismax but i can imagine its a feature. Ths is because inconstent use of stopwords in the analyzers of the fields specified in qf can yield really unexpected results because of the mm parameter. In dismax, if one analyzer removed stopwords and the other doesn't the mm parameter

Re: StopFilterFactory and qf containing some fields that use it and some that do not

2011-01-12 Thread Jayendra Patil
Have used edismax and Stopword filters as well. But usually use the fq parameter e.g. fq=title:the life and never had any issues. Can you turn on the debugQuery and check whats the Query formed for all the combinations you mentioned. Regards, Jayendra On Wed, Jan 12, 2011 at 5:19 PM, Dyer,

Re: StopFilterFactory and qf containing some fields that use it and some that do not

2011-01-12 Thread Markus Jelsma
Have used edismax and Stopword filters as well. But usually use the fq parameter e.g. fq=title:the life and never had any issues. That is because filter queries are not relevant for the mm parameter which is being used for the main query. Can you turn on the debugQuery and check whats the

RE: StopFilterFactory and qf containing some fields that use it and some that do not

2011-01-12 Thread Dyer, James
Here is what debug says each of these queries parse to: 1. q=lifedefType=edismaxqf=Title ... returns 277,635 results 2. q=the lifedefType=edismaxqf=Title ... returns 277,635 results 3. q=lifedefType=edismaxqf=Title Contributor ... returns 277,635 4. q=the lifedefType=edismaxqf=Title Contributor

Re: StopFilterFactory and qf containing some fields that use it and some that do not

2011-01-12 Thread Markus Jelsma
Here's another thread on the subject: http://lucene.472066.n3.nabble.com/Dismax-Minimum-Match-Stopwords-Bug- td493483.html And slightly off topic: you'd also might want to look at using common grams, they are really useful for phrase queries that contain stopwords.