[
https://issues.apache.org/jira/browse/SOLR-17293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18027901#comment-18027901
]
David Smiley commented on SOLR-17293:
-------------------------------------
I noticed that, even with the overseer disabled (aka using "distrib mode"
instead), that not only does an instance of the Overseer exist on each node (I
knew this), it creates instances of ZkDistributedQueue that in turn ensure that
ZK queues exist. [~ilan] do you recall why you left this aspect? Could have
been an unimportant oversight. Perhaps your intention was to ensure that
pre-existing non-processed work gets processed? I don't think that's
critical... someone who asks something of SolrCloud but didn't get a success
response should not assume success. Put differently, layers of a distributed
system need retries.
It occurred to me that the presence of those queues in ZK could be taken to
mean that a cluster should use the Overseer (barring the new cluster prop or
env var, which are more specific & explicit). No need to look at the lowest
Solr version of the cluster to guess. Overseer queues means the Overseer
(duh). This means we need to *stop* creating these ZkDistributedQeue instances
when the Overseer is disabled (albeit still exists as an instance of a Java
class). WDYT [~houston]?
> Umbrella: Decentralized Cluster Processing as default
> -----------------------------------------------------
>
> Key: SOLR-17293
> URL: https://issues.apache.org/jira/browse/SOLR-17293
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
> Reporter: David Smiley
> Priority: Blocker
> Labels: pull-request-available
> Fix For: main (10.0)
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> This is an umbrella issue for tracking work required for running SolrCloud
> with two booleans as default: {{distributedClusterStateUpdates}} and
> {{distributedCollectionConfigSetExecution}}, which we may rename/refactor
> (TBD). When they are set, the Overseer has nothing to do except run "Cluster
> Singleton Plugins" (if you configure any).
> These have been in Solr for years since well before 9.0 and are tested in a
> randomized fashion. But they have not experienced real-world usage to our
> knowledge. There are some scalability concerns, and unclear compatibility
> with PRS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
