RE: Token "states" not getting lemmatized by Solr?

2017-08-11 Thread Markus Jelsma
cene.apache.org > Subject: Re: Token states not getting lemmatized by Solr? > > Hi Omer, > Your analysis chain does not include a stem filter (lemmatizer) > Assuming you are dealing with English text, you can use KStemFilterFactory or > SnowballFilterFactory. > Ahmet > > On T

Re: Token "states" not getting lemmatized by Solr?

2017-08-10 Thread OTH
Hello - Sorry, I obviously made a mistake here. I said earlier that it seems to me that the word 'united' is being lemmatized (to 'unite'). But it seems that's not the case. It seems that there isn't any lemmatization or stemming being done. I had previously assumed that the default

Re: Token "states" not getting lemmatized by Solr?

2017-08-10 Thread Erick Erickson
First, if you turn off the "verbose" checkbox, it'll reduce a lot of the clutter. The important point is that when you hover over those abbreviations, it tells you exactly what class did the associated transformation the analysis chain on the tokens. You'll note that StandardTokenizer breaks the

Re: Token "states" not getting lemmatized by Solr?

2017-08-10 Thread Ahmet Arslan
Hi Omer, Your analysis chain does not include a stem filter (lemmatizer) Assuming you are dealing with English text, you can use KStemFilterFactory or SnowballFilterFactory. Ahmet On Thursday, August 10, 2017, 9:33:08 PM GMT+3, OTH wrote: Hi, Regarding 'analysis

Re: Token "states" not getting lemmatized by Solr?

2017-08-10 Thread OTH
Hi, Regarding 'analysis chain': I'm using Solr 6.4.1, and in the managed-schema file, I find the following: Regarding the Admin UI >> Analysis page: I just tried that, and to be honest, I can't seem to get much useful

Re: Token "states" not getting lemmatized by Solr?

2017-08-10 Thread Erick Erickson
saying the field is "text_general" is not sufficient, please post the analysis chain defined in your schema. Also the admin UI>>analysis page will help you figure out exactly what part of the analysis chain does what. Best, Erick On Thu, Aug 10, 2017 at 8:37 AM, OTH

Token "states" not getting lemmatized by Solr?

2017-08-10 Thread OTH
Hello, It seems for me that the token "states" is not getting lemmatized to "state" by Solr. Eg, I have a document with the value "united states of america". This document is not returned when the following query is issued: q=name:state^1+name:america^1+name:united^1 However, all documents which