Edismax, stopword query and non-existent fields
I've recently had a requirement request from the user to allow a query when
searching for stopwords alone. I discovered that edismax will already do
this for me, all I had to do was remove our StopFilterFactory on the index
analyzer so that the stopwords actually exist in our index.
So suppose I have a collection with 2 fields: an id with type string and a
field called "test" with type text_en. I remove the the StopFilterFactory
from the index analyzer but add it to the query analzyer. I then create 2
documents as below:
{ "id": 1, "test": "and" }
{ "id": 2, "test": "foo" }
"and" is a stopword. The below query only returns the document with id: 2
as expected:
http://localhost:8983/solr/test/select?defType=edismax=test=and+foo
The below query returns only our document with id: 1 as expected:
http://localhost:8983/solr/test/select?defType=edismax=test=and
So far so good.
We also have a requirement to allow partial searching on only some fields.
So some of our fields have an ngram equivalent, we postfix these fields
with "_ngram". So we'd sometimes end up with two fields: fieldname and
fieldname_ngram. We do this for all our fields, even the ones that don't
have an ngram equivalent, so the query will sometimes contain an fq with a
non-existent field:
http://localhost:8983/solr/test/select?defType=edismax=test=test_ngram=foo
This returns our id: 2 document, great! Notice that there's the
non-existent test_ngram added to a qf parameter. However, if I run a query
with a stopword:
http://localhost:8983/solr/test/select?defType=edismax=test=test_ngram=and
Then no documents are returned. This is inconvenient for us. It would be
better if edismax was liberal with non-existent fields. It also seems
inconsistent with the below query which will return the document with id: 1:
http://localhost:8983/solr/test/select?defType=edismax=test=test_ngram=and=false
Would it be possible to ignore non-existent fields when a query only
contains stopwords? Or is there a good reason why this can't be implemented?
Regards,
Craig
Re: ClassCastException when using HighFrequencyDictionaryFactory with SearchComponent
Woops. Thanks Rick, that was it. I'm so used to typing "str" that it became white noise. On 29 August 2017 at 16:03, Rick Leir <rl...@leirtech.com> wrote: > Craig, > Just a guess, but your threshold is in a string field. Cheers -- Rick > > On August 29, 2017 7:37:56 AM MDT, Craig Smiles <smile...@gmail.com> > wrote: > >Hi, > > > >I've been trying to get a suggester working with the > >HighFrequencyDictionaryFactory but when setting the threshold I'm > >getting a > >ClassCastException when the dictionary is built. > > > >Could someone please advise? > > > >To reproduce: > > > >1. Download solr 6.6 > > > >2. Replace the SuggestComponent within the techproducts' solrconfig.xml > >with: > > > > > > mySuggester > > FuzzyLookupFactory > > HighFrequencyDictionaryFactory > > 0.005 > > cat > > string > > false > > > > > > > >3. Run the techproducts example: "solr start -e techproducts" > > > >4. Visit > >http://localhost:8983/solr/techproducts/suggest?suggest= > true=true=mySuggester=elec > > > >I then see the following response: > > > > > > > >500 > >5 > > > > > >java.lang.String cannot be cast to > >java.lang.Float > > > >java.lang.ClassCastException: java.lang.String cannot be cast to > >java.lang.Float at > >org.apache.solr.spelling.suggest.HighFrequencyDictionaryFactory.create( > HighFrequencyDictionaryFactory.java:47) > >at > >org.apache.solr.spelling.suggest.SolrSuggester.build( > SolrSuggester.java:179) > >at > >org.apache.solr.handler.component.SuggestComponent. > prepare(SuggestComponent.java:179) > >at > >org.apache.solr.handler.component.SearchHandler.handleRequestBody( > SearchHandler.java:270) > >at > >org.apache.solr.handler.RequestHandlerBase.handleRequest( > RequestHandlerBase.java:173) > >at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477) at > >org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723) at > >org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529) at > >org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:361) > >at > >org.apache.solr.servlet.SolrDispatchFilter.doFilter( > SolrDispatchFilter.java:305) > >at > >org.eclipse.jetty.servlet.ServletHandler$CachedChain. > doFilter(ServletHandler.java:1691) > >at > >org.eclipse.jetty.servlet.ServletHandler.doHandle( > ServletHandler.java:582) > >at > >org.eclipse.jetty.server.handler.ScopedHandler.handle( > ScopedHandler.java:143) > >at > >org.eclipse.jetty.security.SecurityHandler.handle( > SecurityHandler.java:548) > >at > >org.eclipse.jetty.server.session.SessionHandler. > doHandle(SessionHandler.java:226) > >at > >org.eclipse.jetty.server.handler.ContextHandler. > doHandle(ContextHandler.java:1180) > >at > >org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) > >at > >org.eclipse.jetty.server.session.SessionHandler. > doScope(SessionHandler.java:185) > >at > >org.eclipse.jetty.server.handler.ContextHandler. > doScope(ContextHandler.java:1112) > >at > >org.eclipse.jetty.server.handler.ScopedHandler.handle( > ScopedHandler.java:141) > >at > >org.eclipse.jetty.server.handler.ContextHandlerCollection.handle( > ContextHandlerCollection.java:213) > >at > >org.eclipse.jetty.server.handler.HandlerCollection. > handle(HandlerCollection.java:119) > >at > >org.eclipse.jetty.server.handler.HandlerWrapper.handle( > HandlerWrapper.java:134) > >at > >org.eclipse.jetty.rewrite.handler.RewriteHandler.handle( > RewriteHandler.java:335) > >at > >org.eclipse.jetty.server.handler.HandlerWrapper.handle( > HandlerWrapper.java:134) > >at org.eclipse.jetty.server.Server.handle(Server.java:534) at > >org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at > >org.eclipse.jetty.server.HttpConnection.onFillable( > HttpConnection.java:251) > >at > >org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded( > AbstractConnection.java:273) > >at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at > >org.eclipse.jetty.io.SelectChannelEndPoint$2.run( > SelectChannelEndPoint.java:93) > >at > >org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume. > executeProduceConsume(ExecuteProduceConsume.java:303) > >at > >org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume. > produceConsume(ExecuteProduceConsume.java:148) > >at > >org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run( > ExecuteProduceConsume.java:136) > >at > >org.eclipse.jetty.util.thread.QueuedThreadPool. > runJob(QueuedThreadPool.java:671) > >at > >org.eclipse.jetty.util.thread.QueuedThreadPool$2.run( > QueuedThreadPool.java:589) > >at java.lang.Thread.run(Thread.java:745) > > > >500 > > > > > > > >Thanks, > >Craig > > -- > Sorry for being brief. Alternate email is rickleir at yahoo dot com
ClassCastException when using HighFrequencyDictionaryFactory with SearchComponent
Hi, I've been trying to get a suggester working with the HighFrequencyDictionaryFactory but when setting the threshold I'm getting a ClassCastException when the dictionary is built. Could someone please advise? To reproduce: 1. Download solr 6.6 2. Replace the SuggestComponent within the techproducts' solrconfig.xml with: mySuggester FuzzyLookupFactory HighFrequencyDictionaryFactory 0.005 cat string false 3. Run the techproducts example: "solr start -e techproducts" 4. Visit http://localhost:8983/solr/techproducts/suggest?suggest=true=true=mySuggester=elec I then see the following response: 500 5 java.lang.String cannot be cast to java.lang.Float java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Float at org.apache.solr.spelling.suggest.HighFrequencyDictionaryFactory.create(HighFrequencyDictionaryFactory.java:47) at org.apache.solr.spelling.suggest.SolrSuggester.build(SolrSuggester.java:179) at org.apache.solr.handler.component.SuggestComponent.prepare(SuggestComponent.java:179) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:270) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 500 Thanks, Craig
