[
https://issues.apache.org/jira/browse/OAK-4966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4966:
---------------------------------
Attachment: OAK-4966.patch
proposed patch. includes following changes:
* added maxSize to RecordCache
* added weight (current and max) info to RecordCacheStats based on number of
items and avg size of item, see below
* added guesstimate for avg entry in the cache for the 3 types we cache now:
strings (90% of the max size a string can have in the cache), templates (250
bytes) and nodes (250 bytes).
This adds up to a roughly needed heap of {{3GB}} for compaction (all caches
sizes combined). and would not allow anyone to run it if the resources are not
there.
For now, this is a single check _before_ starting compaction, so not an ongoing
background operation. The downside is that the heap footprint can change
depending on background operations. this was the way it was implemented before
and it seemed good enough. Not sure if this needs to change to provide a more
pro-active barrier, but we can revisit this later if needed.
[~mduerig], [~frm] feedback appreciated!
> Re-introduce a blocker for compaction based on available heap
> -------------------------------------------------------------
>
> Key: OAK-4966
> URL: https://issues.apache.org/jira/browse/OAK-4966
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
> Reporter: Alex Parvulescu
> Assignee: Alex Parvulescu
> Fix For: Segment Tar 0.0.18
>
> Attachments: OAK-4966.patch
>
>
> As seen in a local test, running compaction on a tight heap can lead to
> OOMEs. There used to be a best effort barrier against this situation 'not
> enough heap for compaction', but we removed it with the compaction maps.
> I think it makes sense to add it again based on the max size of some of the
> caches: segment cache {{256MB}} by default [0] and some writer caches which
> can go up to {{2GB}} all combined [1] and probably others I missed.
> [0]
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/SegmentCache.java#L48
> [1]
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/WriterCacheManager.java#L50
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
