[
https://issues.apache.org/jira/browse/SOLR-10506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15972465#comment-15972465
]
ASF GitHub Bot commented on SOLR-10506:
---------------------------------------
GitHub user tboeghk opened a pull request:
https://github.com/apache/lucene-solr/pull/190
[SOLR-10506] Fixes a memory leak in zk schema watching
Upon manual Solr Collection reloading, references to the closed `SolrCore`
are not fully removed by the garbage collector as a strong reference to the
`ZkIndexSchemaReader` is held in a ZooKeeper `Watcher` that watches for schema
changes.
In our case, this leads to a massive memory leak as managed resources are
still referenced by the closed `SolrCore`. Our Solr cloud environment utilizes
rather large managed resources (synonyms, stopwords). To reproduce, we fired
out environment up and reloaded the collection 13 times. As a result we fully
exhausted our heap. A closer look with the Yourkit profiler revealed 13
`SolrCore` instances, still holding strong references to the garbage collection
root.
Each `SolrCore` instance holds a single path with strong references to the
gc root via a `Watcher` in `ZkIndexSchemaReader`. The `ZkIndexSchemaReader`
registers a close hook in the `SolrCore` but the Zookeeper is not removed upon
core close.
We supplied this Github Pull Request that extracts the zookeeper `Watcher`
as a static inner class. To eliminate the memory leak, the schema reader is
held inside a `WeakReference` and the reference is explicitly removed on core
close.
Initially I wanted to supply a test case but unfortunately did not find a
good starting point ...
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shopping24/lucene-solr branch_6_5
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/lucene-solr/pull/190.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #190
----
commit 9a5abee7e9b431ba0c2e0d432dc33b1919fa4f40
Author: Torsten Bøgh Köster <[email protected]>
Date: 2017-04-18T09:43:52Z
Fixes a memory leak in zk schema watching
----
> Possible memory leak upon collection reload
> -------------------------------------------
>
> Key: SOLR-10506
> URL: https://issues.apache.org/jira/browse/SOLR-10506
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Server
> Affects Versions: 6.5
> Reporter: Torsten Bøgh Köster
> Attachments: solr_collection_reload_13_cores.png,
> solr_gc_path_via_zk_WatchManager.png
>
>
> Upon manual Solr Collection reloading, references to the closed {{SolrCore}}
> are not fully removed by the garbage collector as a strong reference to the
> {{ZkIndexSchemaReader}} is held in a ZooKeeper {{Watcher}} that watches for
> schema changes.
> In our case, this leads to a massive memory leak as managed resources are
> still referenced by the closed {{SolrCore}}. Our Solr cloud environment
> utilizes rather large managed resources (synonyms, stopwords). To reproduce,
> we fired out environment up and reloaded the collection 13 times. As a result
> we fully exhausted our heap. A closer look with the Yourkit profiler revealed
> 13 {{SolrCore}} instances, still holding strong references to the garbage
> collection root (see screenshot 1).
> Each {{SolrCore}} instance holds a single path with strong references to the
> gc root via a `Watcher` in `ZkIndexSchemaReader` (see screenshot 2). The
> {{ZkIndexSchemaReader}} registers a close hook in the {{SolrCore}} but the
> Zookeeper is not removed upon core close.
> We supplied a Github Pull Request that extracts the zookeeper `Watcher` as a
> static inner class. To eliminate the memory leak, the schema reader is held
> inside a `WeakReference` and the reference is explicitly removed on core
> close.
> Initially I wanted to supply a test case but unfortunately did not find a
> good starting point ...
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
