Re: requestHandler ignoring synonyms

[Jack Krupansky]

Follow the advice you've already been given: 1) switch from the keyword 
tokenizer to the standard tokenizer, 2) get rid of regex replace (for now), 
and otherwise simplify your analyzers as much as possible. Then run a test 
with a simple, consistent example, and review the debugQuery and parsed 
query output to see what actually happens. And focus on using the Solr Admin 
UI Analysis page to see how your analyzers are really behaving.

-- Jack Krupansky
-----Original Message----- 
From: Carlos Alexandro Becker
Sent: Tuesday, March 05, 2013 8:54 AM
To: solr-user
Subject: Re: requestHandler ignoring synonyms

Hi Jack,

Thanks for your answer, and yes, I'm pretty confused.
The thing is: This problem is going on in one of my job applications, and I
must fix it.

Can you give me some tips or links that I should read to clear my mind and
understand it?

Thanks in advance.


On Tue, Mar 5, 2013 at 10:48 AM, Jack Krupansky 
<j...@basetechnology.com>wrote:

You're getting the 400 error because you are using the keyword tokenizer
which means that there will be lots of "terms" (really just raw strings"
that begin with "java". That simply isn't going to work. Stick with the
standard tokenizer.

You have way too much going on here that is clearly way beyond your
current skill level. You need to back way off, way, way off, and focus on
doing simple things, getting simple things to work and then build
incrementally.

Get rid of the reg ex pattern replace filter as well. Maybe you will
ultimately need it, but at this stage of your confusion it only adds to
your confusion.


-- Jack Krupansky

-----Original Message----- From: Carlos Alexandro Becker
Sent: Tuesday, March 05, 2013 8:33 AM

To: solr-user
Subject: Re: requestHandler ignoring synonyms

Thanks for your answer Erik!

I changed the FieldType to:
https://gist.github.com/**caarlos0/89b7c0484b154550bc63<https://gist.github.com/caarlos0/89b7c0484b154550bc63>

And got a 400 error with message "analyzer returned too many terms for
multiTerm term: java".

I also tried to change the query to do not use wildcard, but it still
ignoring the synonyms..

Any idea?

Thanks


On Tue, Mar 5, 2013 at 10:18 AM, Erik Hatcher <erik.hatc...@gmail.com
>wrote:

 In that last example you're doing a wildcard query (java*), and by 
default
that does not run (all of) the analysis chain you have defined.

If you need to expand synonyms for wildcarded terms like this, you'll 
need
to define a "multiterm" analysis chain.  See here for more details: <
http://wiki.apache.org/solr/**MultitermQueryAnalysis<http://wiki.apache.org/solr/MultitermQueryAnalysis>
>

        Erik


On Mar 5, 2013, at 07:39 , Carlos Alexandro Becker wrote:

> Hi Jack, I've updated the gist:
> 
https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
>
> I'm give you the wrong browser tab result yesterday, sorry.
>
> The schema seems right to me. I have a field name BoosterField, with 
> the
> synonyms and etc enabled in its fieldtype...
>
> can't figure out what's wrong. :S
>
>
> Jack, thanks for your help.
>
>
> On Mon, Mar 4, 2013 at 7:00 PM, Jack Krupansky <j...@basetechnology.com
>wrote:
>
>> Where it says:
>>
>> "querystring":"****DocumentContent:java",
>> "parsedquery":"(+((****DocumentContent:java DocumentContent:notare
>> DocumentContent:jre)~2/no_****coord) () () () () ())/no_coord",
>>
>> That indicates that "java" was expanded to be equivalent to "java",
>> "notare", or "jre".
>>
>> Are you sure you have documents that have "notare" or "jre" in the
>> DocumentContent field?
>>
>> Suggestion: leave off the "DocumentContent:" field from the query 
>> since
it
>> is included in the qf parameter.
>>
>>
>> -- Jack Krupansky
>>
>> -----Original Message----- From: Carlos Alexandro Becker
>> Sent: Monday, March 04, 2013 4:09 PM
>> To: solr-user
>> Subject: Re: requestHandler ignoring synonyms
>>
>>
>> I've done this, but I'm not a specialist, so, I see nothing >>
interesting.
>>
>> The log is: 
>> https://gist.github.com/****caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec>
<
https://gist.github.com/**caarlos0/4ad53583fb2b30ef0bec<https://gist.github.com/caarlos0/4ad53583fb2b30ef0bec>
>
>>
>> Thanks.
>>
>>
>> On Mon, Mar 4, 2013 at 5:05 PM, Jack Krupansky <
j...@basetechnology.com
>**
>> wrote:
>>
>> You can simply test whether synonyms are being ignored or how they are
>>> being processed by using the Solr Admin UI Analysis page. Select the
field
>>> and enter the text to test. It will show you exactly what the synonym
>>> filter does.
>>>
>>> Make sure the synonym file is exactly as specified in the token >>>
filter.
>>>
>>> If there are still problems, show us the field name, field type with
>>> analyzer, and the query response with &debugQuery=true. The parsed
query
>>> string should have query-time synonyms expanded.
>>>
>>> -- Jack Krupansky
>>>
>>> -----Original Message----- From: Carlos Alexandro Becker
>>> Sent: Monday, March 04, 2013 2:16 PM
>>> To: solr-user
>>> Subject: requestHandler ignoring synonyms
>>>
>>>
>>> I have a request handler like this:
>>>
>>> <requestHandler name="/defaultSearch" class="solr.SearchHandler">
>>>   <!-- Adds filters to all queries-->
>>>   <lst name="appends">
>>>     <str name="fq">-DELETED:true</str>
>>>   </lst>
>>>
>>>   <lst name="defaults">
>>>     <!-- See 
>>> http://wiki.apache.org/solr/******DisMaxQParserPlugin<http://wiki.apache.org/solr/****DisMaxQParserPlugin>
<
http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>
>>> 
<http://**wiki.apache.org/**solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
<
http://wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>
>>>> -->
>>>
>>>     <str name="defType">edismax</str>
>>>     <str name="echoParams">explicit</******str>
>>>
>>>     <str name="df">text</str>
>>>     <!-- As dismax is used, 0.00 = is pure dismax , 1.00 = sum -->
>>>     <float name="tie">1.00</float>
>>>     <!-- Query Fields - Search fileds with its height -->
>>>     <str name="qf">
>>>       DocumentDisplayName^10 DocumentName^9
>>>       DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>>>     </str>
>>>     <!-- Phrase Field - frases fields used on quer as phrase >> 
>>> "" -->
>>>     <str name="pf">
>>>       DocumentDisplayName^10 DocumentName^9
>>>       DocumentContent^5 DocumentDescription^3 DocumentTAGS^8
>>>     </str>
>>>     <!-- See 
>>> http://wiki.apache.org/solr/******DisMaxQParserPlugin<http://wiki.apache.org/solr/****DisMaxQParserPlugin>
<
http://wiki.apache.org/solr/****DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
>
>>> 
<http://**wiki.apache.org/**solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/**DisMaxQParserPlugin>
<
http://wiki.apache.org/solr/**DisMaxQParserPlugin<http://wiki.apache.org/solr/DisMaxQParserPlugin>
>
>>>> -->
>>>
>>>     <str name="mm">
>>>       2&lt;-1 5&lt;-2 6&lt;90%
>>>     </str>
>>>     <!-- Phrase whitespace ignore -->
>>>     <int name="ps">10</int>
>>>   </lst>
>>> </requestHandler>
>>>
>>>
>>> but the synonyms and stopwords are ignored. If I use the default
/select
>>> query it works just as expected.
>>>
>>> What can be wrong?
>>>
>>> --
>>> Atenciosamente,
>>> *Carlos Alexandro Becker*
>>> 
http://caarlos0.github.com/******about<http://caarlos0.github.com/****about><
http://caarlos0.github.com/****about <http://caarlos0.github.com/**about>
><
>>> http://caarlos0.github.com/****about<http://caarlos0.github.com/**about><
http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>>
>>>
>>>
>>
>>
>> --
>> Atenciosamente,
>> *Carlos Alexandro Becker*
>> http://caarlos0.github.com/****about<http://caarlos0.github.com/**about><
http://caarlos0.github.com/**about <http://caarlos0.github.com/about>>
>>
>
>
>
> --
> Atenciosamente,
> *Carlos Alexandro Becker*
> http://caarlos0.github.com/**about <http://caarlos0.github.com/about>




--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/**about <http://caarlos0.github.com/about>




--
Atenciosamente,
*Carlos Alexandro Becker*
http://caarlos0.github.com/about