Unless I'm missing something, there is currently no mechanism for a CloudSolrClient and/or HttpShardHandler to be told "retry this collection specific request a max of N times" in the event that there are more then N replicas of the collection/shard that LBSolrClient could send this request to.
(We *do* have MAX_STALE_RETRIES - but that is only for stale state situations, and has no interaction on the "try next url" loop inside LBSolrClient.request(...))
So let's suppose the existence of some "query of death" that could cause a solr thread to crash (consume all the heap, or call System.exit, or some other catastrophe, etc...) Ideally this doesn't exist, but let's think defensively.
If you have a huge cluster of solr nodes, each hosting a replica of collectionX, your front end application probably uses CloudSolrClient w/LBSolrClient to talk to this cluster so you have some resilency in case a particular node/replica is down (and you can take advantage of the LBSolrClient "zombie tracking").
But now you have a situation where if CloudSolrClient can be tricked into sending this query of death to *every* solr node in sequence -- because the first node it's sent to will crash, so then LBSolrClient will say "huh, replica1 closed it's socket (i wonder why?) i guess i should mark it as a zombie and move on to replica2." ... and this process will just keep repeating.
In theory, the ReplicaListTransformer API can not only transform the "order" (sorted or shuffled or etc.) of the replicas in it's input, but also "filter" the list replicas (Example: just the first N replicas after being re-ordered) but in practice there is no actual option to do this, and the ReplicaListTransformerGenerator isn't actually a "plugin" that supports customized implementations (in neither the slr server, nor SolrJ usages)
So.... should there be a mechanism to configure CloudSolrClient & HttpShardHandler to limit the total number of nodes/replicas that any request should be sent to? and/or a "max retries" type option on SolrRequests? (but if so, how should that "per-request" limit interact with internal per-shard requests that may spawn from it?)
-Hoss http://www.lucidworks.com/ --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
