[
https://issues.apache.org/jira/browse/SOLR-17895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18130690#comment-18130690
]
Amrit Sarkar commented on SOLR-17895:
-------------------------------------
[~dsmiley] I looked at what a switch would involve:
* Curator's lock driver already creates parents with
{{creatingParentContainersIfNeeded}}, so a switch would replace
{{makeLockPath}} from this PR. The DELETE cleanup stays needed either way:
persistent dirs left by older versions don't go away on their own, whatever the
lock implementation.
* {{curator-recipes}} isn't a Solr dependency yet, only {{curator-framework}}
and {{curator-client}}.
* Mirroring is the real question. A sub-command adopts its caller's existing
lock node at each level ({{callingLockId}} passed as {{lockIdToMirror}}), and
release leaves that node alone. {{InterProcessReadWriteLock}} only supports
per-thread reentrancy, and none of its public constructors take a custom
driver, so adopting a node means extending Curator's internals or keeping
mirroring outside Curator.
[PR #5161|https://github.com/apache/solr/pull/5161] fixes the leak now and
should land as is. After it merges I'll pick up SOLR-17704 and start with
mirroring. If that has no clean answer, I'll write up why on the issue.
> Persistent ZNodes are leaked when running Solr Cloud with distributed
> collection commands
> -----------------------------------------------------------------------------------------
>
> Key: SOLR-17895
> URL: https://issues.apache.org/jira/browse/SOLR-17895
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
> Affects Versions: 9.0, 10.0
> Reporter: Paul McArthur
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When a collection command is executed in distributed mode
> (DistributedCollectionCommandRunner), an ephemeral ZNode is created to lock
> the collection, shard or replica that prevents other commands from executing
> concurrently and potentially conflicting with each other.
>
> The ephemeral nodes are created in a tree structure under
> {_}*distributedapi/collectionlocks*{_}, with persistent znodes created for
> collName
> collName/Locks
> collName/_shard
> collName/_shard/Locks
> collName/_shard/_replica
>
> These nodes are never deleted from ZooKeeper, even when the collection is
> deleted, and can therefore accrue over time.
> In our specific case, we have accumulated many thousands of these nodes over
> a few months. While there appears to be no direct impact on performance,
> these nodes should nonetheless be removed when they are no longer required.
>
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
