RE: Wordbreak spellchecker excessive breaking.
I am not sure why changing spellcheck parameters would prevent your server from restarting. One thing to check is to see if you have warming queries running that involve spellcheck. I think I remember from long ago there was (maybe still is) an obscure bug where sometimes it will lock up in rare cases when spellcheck is used in warming queries. I do not remember exactly what caused this or if it was ever fixed. Besides that, you might want to post a stack trace or describe what happens when it doesn't restart. Perhaps someone here will know what the problem is. James Dyer Ingram Content Group (615) 213-4311 -Original Message- From: S.L [mailto:simpleliving...@gmail.com] Sent: Friday, May 30, 2014 12:36 AM To: solr-user@lucene.apache.org Subject: Re: Wordbreak spellchecker excessive breaking. James, Thanks for clearly stating this , I was not able to find this documented anywhere, yes I am using it with another spell checker (Direct) with the collation on. I will try the maxChangtes and let you know. On a side note , whenever I change the spellchecker parameter , I need to rebuild the index and delete the solr data directory before that as my Tomcat instance would not even start, can you let me know why ? Thanks. On Tue, May 27, 2014 at 12:21 PM, Dyer, James james.d...@ingramcontent.com wrote: You can do this if you set it up like in the mail Solr example: lst name=spellchecker str name=namewordbreak/str str name=classnamesolr.WordBreakSolrSpellChecker/str str name=fieldname/str str name=combineWordstrue/str str name=breakWordstrue/str int name=maxChanges10/int /lst The combineWords and breakWords flags let you tell it which kind of workbreak correction you want. maxChanges controls the maximum number of words it can break 1 word into, or the maximum number of words it can combine. It is reasonable to set this to 1 or 2. The best way to use this is in conjunction with a regular spellchecker like DirectSolrSpellChecker. When used together with the collation functionality, it should take a query like mob ile and depending on what actually returns results from your data, suggest either mobile or perhaps mob lie or both. The one thing is cannot do is fix a transposition or misspelling and combine or break words in one shot. That is, it cannot detect that mob lie should become mobile. James Dyer Ingram Content Group (615) 213-4311 -Original Message- From: S.L [mailto:simpleliving...@gmail.com] Sent: Saturday, May 24, 2014 4:21 PM To: solr-user@lucene.apache.org Subject: Wordbreak spellchecker excessive breaking. I am using Solr wordbreak spellchecker and the issue is that when I search for a term like mob ile expecting that the wordbreak spellchecker would actually resutn a suggestion for mobile it breaks the search term into letters like m o b I have two issues with this behavior. 1. How can I make Solr combine mob ile to mobile? 2. Not withstanding the fact that my search term mob ile is being broken incorrectly into individual letters , I realize that the wordbreak is needed in certain cases, how do I control the wordbreak so that it does not break it into letters like m o b which seems like excessive breaking to me ? Thanks.
Re: Wordbreak spellchecker excessive breaking.
-- !-- Result Window Size An optimization for use with the queryResultCache. When a search is requested, a superset of the requested number of document ids are collected. For example, if a search for a particular query requests matching documents 10 through 19, and queryWindowSize is 50, then documents 0 through 49 will be collected and cached. Any further requests in that range can be satisfied via the cache. -- queryResultWindowSize20/queryResultWindowSize !-- Maximum number of documents to cache for any entry in the queryResultCache. -- queryResultMaxDocsCached200/queryResultMaxDocsCached !-- Query Related Event Listeners Various IndexSearcher related events can trigger Listeners to take actions. newSearcher - fired whenever a new searcher is being prepared and there is a current searcher handling requests (aka registered). It can be used to prime certain caches to prevent long request times for certain requests. firstSearcher - fired whenever a new searcher is being prepared but there is no current registered searcher to handle requests or to gain autowarming data from. -- !-- QuerySenderListener takes an array of NamedList and executes a local query request for each NamedList in sequence. -- listener event=newSearcher class=solr.QuerySenderListener arr name=queries !-- lststr name=qsolr/strstr name=sortprice asc/str/lst lststr name=qrocks/strstr name=sortweight asc/str/lst -- /arr /listener listener event=firstSearcher class=solr.QuerySenderListener arr name=queries lst str name=qstatic firstSearcher warming in solrconfig.xml/str /lst /arr /listener !-- Use Cold Searcher If a search request comes in and there is no current registered searcher, then immediately register the still warming searcher and use it. If false then all requests will block until the first searcher is done warming. -- useColdSearcherfalse/useColdSearcher !-- Max Warming Searchers Maximum number of searchers that may be warming in the background concurrently. An error is returned if this limit is exceeded. Recommend values of 1-2 for read-only slaves, higher for masters w/o cache warming. -- maxWarmingSearchers2/maxWarmingSearchers /query On Fri, May 30, 2014 at 10:20 AM, Dyer, James james.d...@ingramcontent.com wrote: I am not sure why changing spellcheck parameters would prevent your server from restarting. One thing to check is to see if you have warming queries running that involve spellcheck. I think I remember from long ago there was (maybe still is) an obscure bug where sometimes it will lock up in rare cases when spellcheck is used in warming queries. I do not remember exactly what caused this or if it was ever fixed. Besides that, you might want to post a stack trace or describe what happens when it doesn't restart. Perhaps someone here will know what the problem is. James Dyer Ingram Content Group (615) 213-4311 -Original Message- From: S.L [mailto:simpleliving...@gmail.com] Sent: Friday, May 30, 2014 12:36 AM To: solr-user@lucene.apache.org Subject: Re: Wordbreak spellchecker excessive breaking. James, Thanks for clearly stating this , I was not able to find this documented anywhere, yes I am using it with another spell checker (Direct) with the collation on. I will try the maxChangtes and let you know. On a side note , whenever I change the spellchecker parameter , I need to rebuild the index and delete the solr data directory before that as my Tomcat instance would not even start, can you let me know why ? Thanks. On Tue, May 27, 2014 at 12:21 PM, Dyer, James james.d...@ingramcontent.com wrote: You can do this if you set it up like in the mail Solr example: lst name=spellchecker str name=namewordbreak/str str name=classnamesolr.WordBreakSolrSpellChecker/str str name=fieldname/str str name=combineWordstrue/str str name=breakWordstrue/str int name=maxChanges10/int /lst The combineWords and breakWords flags let you tell it which kind of workbreak correction you want. maxChanges controls the maximum number of words it can break 1 word into, or the maximum number of words it can combine. It is reasonable to set this to 1 or 2. The best way to use this is in conjunction with a regular spellchecker like DirectSolrSpellChecker. When used together with the collation functionality, it should take a query like mob ile and depending on what actually returns results from your data, suggest either mobile or perhaps mob lie or both. The one thing is cannot do is fix
Re: Wordbreak spellchecker excessive breaking.
James, Thanks for clearly stating this , I was not able to find this documented anywhere, yes I am using it with another spell checker (Direct) with the collation on. I will try the maxChangtes and let you know. On a side note , whenever I change the spellchecker parameter , I need to rebuild the index and delete the solr data directory before that as my Tomcat instance would not even start, can you let me know why ? Thanks. On Tue, May 27, 2014 at 12:21 PM, Dyer, James james.d...@ingramcontent.com wrote: You can do this if you set it up like in the mail Solr example: lst name=spellchecker str name=namewordbreak/str str name=classnamesolr.WordBreakSolrSpellChecker/str str name=fieldname/str str name=combineWordstrue/str str name=breakWordstrue/str int name=maxChanges10/int /lst The combineWords and breakWords flags let you tell it which kind of workbreak correction you want. maxChanges controls the maximum number of words it can break 1 word into, or the maximum number of words it can combine. It is reasonable to set this to 1 or 2. The best way to use this is in conjunction with a regular spellchecker like DirectSolrSpellChecker. When used together with the collation functionality, it should take a query like mob ile and depending on what actually returns results from your data, suggest either mobile or perhaps mob lie or both. The one thing is cannot do is fix a transposition or misspelling and combine or break words in one shot. That is, it cannot detect that mob lie should become mobile. James Dyer Ingram Content Group (615) 213-4311 -Original Message- From: S.L [mailto:simpleliving...@gmail.com] Sent: Saturday, May 24, 2014 4:21 PM To: solr-user@lucene.apache.org Subject: Wordbreak spellchecker excessive breaking. I am using Solr wordbreak spellchecker and the issue is that when I search for a term like mob ile expecting that the wordbreak spellchecker would actually resutn a suggestion for mobile it breaks the search term into letters like m o b I have two issues with this behavior. 1. How can I make Solr combine mob ile to mobile? 2. Not withstanding the fact that my search term mob ile is being broken incorrectly into individual letters , I realize that the wordbreak is needed in certain cases, how do I control the wordbreak so that it does not break it into letters like m o b which seems like excessive breaking to me ? Thanks.
RE: Wordbreak spellchecker excessive breaking.
You can do this if you set it up like in the mail Solr example: lst name=spellchecker str name=namewordbreak/str str name=classnamesolr.WordBreakSolrSpellChecker/str str name=fieldname/str str name=combineWordstrue/str str name=breakWordstrue/str int name=maxChanges10/int /lst The combineWords and breakWords flags let you tell it which kind of workbreak correction you want. maxChanges controls the maximum number of words it can break 1 word into, or the maximum number of words it can combine. It is reasonable to set this to 1 or 2. The best way to use this is in conjunction with a regular spellchecker like DirectSolrSpellChecker. When used together with the collation functionality, it should take a query like mob ile and depending on what actually returns results from your data, suggest either mobile or perhaps mob lie or both. The one thing is cannot do is fix a transposition or misspelling and combine or break words in one shot. That is, it cannot detect that mob lie should become mobile. James Dyer Ingram Content Group (615) 213-4311 -Original Message- From: S.L [mailto:simpleliving...@gmail.com] Sent: Saturday, May 24, 2014 4:21 PM To: solr-user@lucene.apache.org Subject: Wordbreak spellchecker excessive breaking. I am using Solr wordbreak spellchecker and the issue is that when I search for a term like mob ile expecting that the wordbreak spellchecker would actually resutn a suggestion for mobile it breaks the search term into letters like m o b I have two issues with this behavior. 1. How can I make Solr combine mob ile to mobile? 2. Not withstanding the fact that my search term mob ile is being broken incorrectly into individual letters , I realize that the wordbreak is needed in certain cases, how do I control the wordbreak so that it does not break it into letters like m o b which seems like excessive breaking to me ? Thanks.
Re: Wordbreak spellchecker excessive breaking.
Anyone ? On Sat, May 24, 2014 at 5:21 PM, S.L simpleliving...@gmail.com wrote: I am using Solr wordbreak spellchecker and the issue is that when I search for a term like mob ile expecting that the wordbreak spellchecker would actually resutn a suggestion for mobile it breaks the search term into letters like m o b I have two issues with this behavior. 1. How can I make Solr combine mob ile to mobile? 2. Not withstanding the fact that my search term mob ile is being broken incorrectly into individual letters , I realize that the wordbreak is needed in certain cases, how do I control the wordbreak so that it does not break it into letters like m o b which seems like excessive breaking to me ? Thanks.
Wordbreak spellchecker excessive breaking.
I am using Solr wordbreak spellchecker and the issue is that when I search for a term like mob ile expecting that the wordbreak spellchecker would actually resutn a suggestion for mobile it breaks the search term into letters like m o b I have two issues with this behavior. 1. How can I make Solr combine mob ile to mobile? 2. Not withstanding the fact that my search term mob ile is being broken incorrectly into individual letters , I realize that the wordbreak is needed in certain cases, how do I control the wordbreak so that it does not break it into letters like m o b which seems like excessive breaking to me ? Thanks.
