OK, but the consider what you can do to keep from having to
create such things.
1> for infix notation (leading and trailing wildcards) you can
use ngrams to turn them into simple queries. These are
performance-killers.
2> Use reverseWildcardFactory to deal with leading wildcards
3> restrict the wildcard search to at least two "real" characters.
I can (and have) argued that a* is not useful to a user at all.
The point is that your responsiveness will suffer if you come
anywhere near 100K OR clauses. Maybe that's OK, but
it's lurking out there.
And no I can't see the image, the Apache server is pretty
aggressive about stripping those too.
Best,
Erick
On Tue, Jun 28, 2016 at 7:19 AM, Pablo Anzorena <anzorena.f...@gmail.com>
wrote:
> Hi Erick, thanks for answering.
>
> I attached the image to the body so you can see it.
>
>
>
> Why do I need so many clauses?
> It is because I have two text fields that contains in average 25 words
> with a lot of typos (which I'm not cleaning it) and on top of that the
> index consists of 25 million records. And we let the users make queries
> with phrases, wildcards prefix and suffix. So for example the following
> query is valid *q=text_field:"*ban*" AND text_field2:"foo* bar*"* (I
> ommited the query parser syntax that allows this).
>
>
> 2016-06-28 2:08 GMT-03:00 Erick Erickson <erickerick...@gmail.com>:
>
>> That error sometimes gets reported inappropriately, as long as the
>> servers are live
>> you can pretty much ignore it.
>>
>> Attachments pretty much all get stripped by the mail server so we can't
>> see your
>> screen shot.
>>
>> Setting your max clause count to over 100K is pretty much an
>> anti-pattern, what in the world
>> are you doing that would require it to be that high? You haven't
>> really shown us the query
>> you're sending, but I bet it's amazing. Frankly, anything over the
>> default of 1K is suspect.
>>
>> If this is some clause like id:(1 OR 2 OR 3 OR 4.......) you really
>> want to try using the
>> TermsQueryParser (note the 's' as opposed to TermQueryParser (no 's').
>> See:
>>
>> https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-TermQueryParser
>>
>> And if you use the TermsQueryParser, it's probably a good idea to sort
>> the list of terms, it's
>> more efficient.
>>
>> Or do a join or... because anything with that many terms will be
>> rather slow to say the least.
>>
>> Best,
>> Erick
>>
>>
>> On Mon, Jun 27, 2016 at 8:38 AM, Pablo Anzorena <anzorena.f...@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > I have an ensemble zookeeper consisting of 3 machines and 2 machines
>> with
>> > solrcloud.
>> >
>> > With a high frequency I see in the logging:
>> > No live SolrServers available to handle this
>> > request:[
>> http://solr2:8983/solr/usa_bills_imp_2016_20160623000000_shard1_replica1,
>> >
>> http://solr3:8983/solr/usa_bills_imp_2016_20160623000000_shard1_replica2]
>> >
>> > and the state.json is:
>> >
>> > {"usa_bills_imp_2016_20160623000000":{
>> > "replicationFactor":"2",
>> > "shards":{"shard1":{
>> > "range":"80000000-7fffffff",
>> > "state":"active",
>> > "replicas":{
>> > "core_node1":{
>> > "core":"usa_bills_imp_2016_20160623000000_shard1_replica2",
>> > "base_url":"http://solr3:8983/solr";,
>> > "node_name":"solr3:8983_solr",
>> > "state":"active"},
>> > "core_node2":{
>> > "core":"usa_bills_imp_2016_20160623000000_shard1_replica1",
>> > "base_url":"http://solr2:8983/solr";,
>> > "node_name":"solr2:8983_solr",
>> > "state":"active",
>> > "leader":"true"}}}},
>> > "router":{"name":"compositeId"},
>> > "maxShardsPerNode":"1",
>> > "autoAddReplicas":"false"}}
>> >
>> > And the full stacktrace of the error is:
>> >
>> > null:org.apache.solr.common.SolrException:
>> > org.apache.solr.client.solrj.SolrServerException: No live SolrServers
>> > available to handle this
>> > request:[
>> http://solr3:8983/solr/usa_bills_imp_2014_20160603115528_shard1_replica2,
>> >
>> http://solr3:8983/solr/usa_bills_imp_2016_20160623000000_shard1_replica2,
>> >
>> http://solr2:8983/solr/usa_bills_imp_2014_20160603115528_shard1_replica1,
>> >
>> http://solr2:8983/solr/usa_bills_imp_2015_20160610125230_shard1_replica1]
>> > at
>> >
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:375)
>> > at
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
>> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
>> > at
>> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
>> > at
>> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
>> > at
>> >
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
>> > at
>> >
>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
>> > at
>> >
>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
>> > at
>> >
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>> > at
>> >
>> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
>> > at
>> >
>> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
>> > at
>> >
>> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
>> > at
>> >
>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
>> > at
>> >
>> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>> > at
>> >
>> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
>> > at
>> >
>> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>> > at
>> >
>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
>> > at
>> >
>> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
>> > at
>> >
>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
>> > at org.eclipse.jetty.server.Server.handle(Server.java:497)
>> > at
>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
>> > at
>> >
>> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
>> > at
>> >
>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
>> > at
>> >
>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
>> > at
>> >
>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
>> > at java.lang.Thread.run(Thread.java:745)
>> > Caused by: org.apache.solr.client.solrj.SolrServerException: No live
>> > SolrServers available to handle this
>> > request:[
>> http://solr3:8983/solr/usa_bills_imp_2014_20160603115528_shard1_replica2,
>> >
>> http://solr3:8983/solr/usa_bills_imp_2016_20160623000000_shard1_replica2,
>> >
>> http://solr2:8983/solr/usa_bills_imp_2014_20160603115528_shard1_replica1,
>> >
>> http://solr2:8983/solr/usa_bills_imp_2015_20160610125230_shard1_replica1]
>> > at
>> >
>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355)
>> > at
>> >
>> org.apache.solr.handler.component.HttpShardHandlerFactory.makeLoadBalancedRequest(HttpShardHandlerFactory.java:246)
>> > at
>> >
>> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:221)
>> > at
>> >
>> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:183)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> > at
>> >
>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> > at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> > ... 1 more
>> > Caused by:
>> > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
>> Error
>> > from server at
>> >
>> http://solr3:8983/solr/usa_bills_imp_2014_20160603115528_shard1_replica2:
>> > maxClauseCount is set to 102400
>> > at
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
>> > at
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
>> > at
>> >
>> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
>> > at
>> >
>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
>> > at
>> >
>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:344)
>> > ... 10 more
>> >
>> > So my question is if it is the maxClauseCount triggering the "No live
>> > SolrServers" or if it is something else.
>> > I attach the Admin UI of one of the SolrServers.
>> > The zookeeper log does not throw any errors regarding to this.
>> >
>> >
>> > Thanks.
>> >
>>
>
>
