[
https://issues.apache.org/jira/browse/SOLR-18298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18108582#comment-18108582
]
Shrey Narayan commented on SOLR-18298:
--------------------------------------
Thanks, Houston — I reworked this along the ownership boundary you suggested. I
reverted the
semantic changes in OnDisconnect and OnReconnect, so those shared adapters
retain their existing
behavior. ZkController and standalone ZkStateReader now each maintain a private
AtomicBoolean that
is set only when OnDisconnect reports sessionExpired=true (Curator LOST). Their
reconnect handlers use
compareAndSet(true, false), so a transient SUSPENDED → RECONNECTED does not run
full recovery,
while LOST → RECONNECTED runs it once and atomically consumes the flag.
I added coverage for the unchanged listener contracts and a three-node
ZooKeeper integration test that
distinguishes ordinary ensemble failover from injected session expiration.
`./gradlew tidy`, the two
focused test targets (2 listener tests + 1 integration test), and `git diff
--check` pass. I also launched the
patched build locally as SolrCloud on a redacted local port with embedded
ZooKeeper on a redacted
local port and verified commit 21cb72bc75 in the Admin UI. This keeps the state
local and temporary, so
it can be removed cleanly when persistent watchers eliminate this recovery path.
> ZkController.onReconnect Is Triggered Excessively
> -------------------------------------------------
>
> Key: SOLR-18298
> URL: https://issues.apache.org/jira/browse/SOLR-18298
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
> Affects Versions: 10.0
> Reporter: Jerry
> Priority: Blocker
> Labels: patch, pull-request-available, zookeeper
> Attachments:
> SOLR-18298_PR-4774_AtomicBoolean_Technical_Brief_Secure_Shrey_Narayan_Public_Citations.pdf,
> SOLR-18298_PR-4774_Technical_Review_NextBricks_Shrey Narayan.pdf
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> Curator's RECONNECTED event is different from the previous RECONNECTED event.
> Before Solr10, the OnReconnect is only triggered after a session expiration
> and reconnection.
> Check the following
> [https://github.com/apache/solr/blob/fdb5314279657f7895a90123436d834e81ea3157/solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ConnectionManager.java#L165]
>
> [https://github.com/apache/solr/blob/fdb5314279657f7895a90123436d834e81ea3157/solr/solrj-zookeeper/src/java/org/apache/solr/common/cloud/ConnectionManager.java#L199]
> But Curator's RECONNECTED event is triggered every time a Solr node is
> disconnected from a ZooKeeper instance and reconnected to another
> ZooKeeper instance
> Therefore, currently ZkController.onReconnect is invoked every time a Solr
> node is disconnected from a ZooKeeper instance and reconnected, which is a
> huge overhead, especially when we need to rolling restart a ZooKeeper
> Cluster. It can take more than 10 minutes for a small Solr cluster to level
> out.
> Similiarly, now ZkController.onDisconnect is triggered just after a
> disconnection from a Zookeeper instance. It should only be triggered after a
> session expiration.
> The PR restores the Solr 9 behavior, triggering re-election and other
> initialization staff only when the session actually expires, which
> significantly reduces performance overhead during network hiccups or
> situations like rolling-restart Zookeeper Cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
