Michael Sun created SOLR-10015:
----------------------------------
Summary: Remove strong reference to Field Cache key (optional) so
that GC can release some Field Cache entries when Solr is under memory pressure
Key: SOLR-10015
URL: https://issues.apache.org/jira/browse/SOLR-10015
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Michael Sun
In current Field Cache (FC) implementation, a WeakHashMap is used, supposedly
designed to allow GC to release some Field Cache entries when Solr is under
memory pressure.
However, in practice, FC entry releasing seldom happens. Even worse, sometime
Solr goes OOM and heap dump shows large amount of memory is actually used by
FC. It's a sign that GC is not able to release FC entries even WeakHashMap is
used.
The reason is that FC is using SegmentCoreReaders as the key to the
WeakHashMap. However, SegmentCoreReaders is usually strong referenced by
SegmentReader. A strong reference would prevent GC to release the key and
therefore the value. Therefore GC can't release entries in FC's WeakHashMap.
The JIRA is to propose a solution to remove the strong reference mentioned
above so that GC can release FC entries to avoid long GC pause or OOM. It needs
to be optional because this change is a tradeoff, trading more CPU cycles for
low memory footage. User can make final decision depending on their use cases.
The prototype attached use a combination of directory name and segment name as
key to the WeakHashMap, replacing the SegmentCoreReaders. Without change, Solr
doesn't release any FC entries after a GC is manually triggered. With the
change, FC entries are usually released after GC.
However, I am not sure if it's the best way to solve this problem. Any
suggestions are welcome.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
