[
https://issues.apache.org/jira/browse/JXPATH-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephan Birkl updated JXPATH-195:
---------------------------------
Description:
JXPathContextReferenceImpl caches already compiled expressions in a static
HashMap. This map holds the compiled expressions as SoftReference. The map has
no size limit but the cleanup strategy is that every 500 puts the map is
scanned whether values have been already collected by gc (soft refs) and in
this case removed from the map completely.
There is also a static feature toggle in code (but not accessible via
configuration) to not use the soft references feature - but in this case the
map is not bounded and not cleaned up at all.
In our use case the SoftReference cleanup strategy has some major disadvantages:
* At least when using Oracle JDK 8, SoftReferences are only cleaned up when a
full GC occurs (for example because of an OOM error)
* Our application uses G1GC that is tuned to basically never perform a full
GC, so a clean up never happens under normal operation
* So waiting for a full GC is definitely too late. For example I've found 3.2
mio entries in a heap dump of our application, taking about 4 GB of old gen
heap memory. Application performance already degraded because this caused
frequent old gen GC activities, but SoftReferences were not cleaned up yet
* This problem is of course application and use case specific - what I want to
point out is that the current cleanup strategy can cause problems depending on
the environment.
Attached is a patch against the current master. It keeps the current behavior
by default but adds / changes the following:
* Adds a system property "jxpath.softcache" to enable or disable the softcache
feature. Default is enabled. It basically makes the already existing static
feature toggle USE_SOFT_CACHE configurable.
* If softcache feature is disabled it uses a bounded LRU Map instead of an
unbounded HashMap to limit the cache size this way. The implementation uses
java builtin LinkedHashMap to not introduce new dependencies.
* The size limit in case of soft cache disables can be configured via system
property "jxpath.cache.size". The default is 100 which is perfect for our use
case but may be too small for other use cases. I'm definitely fine if you
decide that a higher number would be better as default.
was:
JXPathContextReferenceImpl caches already compiled expressions in a static
HashMap_._ This map holds the compiled expressions as SoftReference. The map
has no size limit but the cleanup strategy is that every 500 puts the map is
scanned whether values have been already collected by gc (soft refs) and in
this case removed from the map completely.
There is also a static feature toggle in code (but not accessible via
configuration) to not use the soft references feature - but in this case the
map is not bounded and not cleaned up at all.
In our use case the SoftReference cleanup strategy has some major disadvantages:
* At least when using Oracle JDK 8, SoftReferences are only cleaned up when a
full GC occurs (for example because of an OOM error)
* Our application uses G1GC that is tuned to basically never perform a full
GC, so a clean up never happens under normal operation
* So waiting for a full GC is definitely too late. For example I've found 3.2
mio entries in a heap dump of our application, taking about 4 GB of old gen
heap memory. Application performance already degraded because this caused
frequent old gen GC activities, but SoftReferences were not cleaned up yet
* This problem is of course application and use case specific - what I want to
point out is that the current cleanup strategy can cause problems depending on
the environment.
Attached is a patch against the current master. It keeps the current behavior
by default but adds / changes the following:
* Adds a system property "jxpath.softcache" to enable or disable the softcache
feature. Default is enabled. It basically makes the already existing static
feature toggle USE_SOFT_CACHE configurable.
* If softcache feature is disabled it uses a bounded LRU Map instead of an
unbounded HashMap to limit the cache size this way. The implementation uses
java builtin LinkedHashMap to not introduce new dependencies.
* The size limit in case of soft cache disables can be configured via system
property "jxpath.cache.size". The default is 100 which is perfect for our use
case but may be too small for other use cases. I'm definitely fine if you
decide that a higher number would be better as default.
> JXPathContextReferenceImpl soft cache can cause OOM
> ---------------------------------------------------
>
> Key: JXPATH-195
> URL: https://issues.apache.org/jira/browse/JXPATH-195
> Project: Commons JXPath
> Issue Type: Bug
> Affects Versions: 1.3, 1.4
> Reporter: Stephan Birkl
> Priority: Major
> Labels: patch
> Attachments: JXPathContextReferenceImpl.java.patch
>
>
> JXPathContextReferenceImpl caches already compiled expressions in a static
> HashMap. This map holds the compiled expressions as SoftReference. The map
> has no size limit but the cleanup strategy is that every 500 puts the map is
> scanned whether values have been already collected by gc (soft refs) and in
> this case removed from the map completely.
> There is also a static feature toggle in code (but not accessible via
> configuration) to not use the soft references feature - but in this case the
> map is not bounded and not cleaned up at all.
> In our use case the SoftReference cleanup strategy has some major
> disadvantages:
> * At least when using Oracle JDK 8, SoftReferences are only cleaned up when
> a full GC occurs (for example because of an OOM error)
> * Our application uses G1GC that is tuned to basically never perform a full
> GC, so a clean up never happens under normal operation
> * So waiting for a full GC is definitely too late. For example I've found
> 3.2 mio entries in a heap dump of our application, taking about 4 GB of old
> gen heap memory. Application performance already degraded because this caused
> frequent old gen GC activities, but SoftReferences were not cleaned up yet
> * This problem is of course application and use case specific - what I want
> to point out is that the current cleanup strategy can cause problems
> depending on the environment.
> Attached is a patch against the current master. It keeps the current behavior
> by default but adds / changes the following:
> * Adds a system property "jxpath.softcache" to enable or disable the
> softcache feature. Default is enabled. It basically makes the already
> existing static feature toggle USE_SOFT_CACHE configurable.
> * If softcache feature is disabled it uses a bounded LRU Map instead of an
> unbounded HashMap to limit the cache size this way. The implementation uses
> java builtin LinkedHashMap to not introduce new dependencies.
> * The size limit in case of soft cache disables can be configured via system
> property "jxpath.cache.size". The default is 100 which is perfect for our use
> case but may be too small for other use cases. I'm definitely fine if you
> decide that a higher number would be better as default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
