[
https://issues.apache.org/jira/browse/SOLR-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15903394#comment-15903394
]
albert vico oton commented on SOLR-5872:
----------------------------------------
Already did that, but nodes still notify its state change, apparently
collections need to know about other collections status in order to reroute
queries to them, this amount of state change msg was killing our io in the ZK
cluster, causing a lot of cpu wait time and effectively rendering the system
unusable. But honestly that's as far as we went, we moved away from solrcloud
and now we are using standalone solr inside each container for each of our
collections and doing the balancing between replicas through nginx.
> Eliminate overseer queue
> -------------------------
>
> Key: SOLR-5872
> URL: https://issues.apache.org/jira/browse/SOLR-5872
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: Noble Paul
> Assignee: Noble Paul
>
> The overseer queue is one of the busiest points in the entire system. The
> raison d'être of the queue is
> * Provide batching of operations for the main clusterstate,json so that
> state updates are minimized
> * Avoid race conditions and ensure order
> Now , as we move the individual collection states out of the main
> clusterstate.json, the batching is not useful anymore.
> Race conditions can easily be solved by using a compare and set in Zookeeper.
> The proposed solution is , whenever an operation is required to be performed
> on the clusterstate, the same thread (and of course the same JVM)
> # read the fresh state and version of zk node
> # construct the new state
> # perform a compare and set
> # if compare and set fails go to step 1
> This should be limited to all operations performed on external collections
> because batching would be required for others
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
