[
https://issues.apache.org/jira/browse/OAK-7860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662054#comment-16662054
]
Alex Deparvu commented on OAK-7860:
-----------------------------------
WIP branch is [0]. the changes are pretty straight-forward, but instrumenting
the benchmark to compare was a bit more work.
Results are very encouraging, the OOME doesn't happen anymore (at the cost of a
small perf drop).
{noformat}
[trunk]
# CanReadNonExisting C min 10% 50% 90% max
N
Oak-Segment-Tar 1 992 1060 1199 1322 1391
51
Oak-Segment-Tar 5 1999 2165 2800 3393 4072
110
Oak-Segment-Tar 10 2394 4194 5168 6639 6739
115
{noformat}
{noformat}
[patch]
Oak-Segment-Tar 1 942 956 1104 1160 1249
56
Oak-Segment-Tar 5 1899 1944 2137 2290 2479
145
Oak-Segment-Tar 10 2196 4708 4965 5393 5570
123
Oak-Segment-Tar 50 30233 31458 35693 40658 41242
100
Oak-Segment-Tar 100 63166 64998 65880 66479 66608
100
{noformat}
Trunk crashes at 50 concurrent threads: {{java.lang.OutOfMemoryError: Java heap
space}}, while the patched version is very robust (see also the heap
screenshots).
[0]
https://github.com/apache/jackrabbit-oak/compare/trunk...stillalex:oome-permissionentrycache
> Make PermissionEntryCache more resilient against OOME
> -----------------------------------------------------
>
> Key: OAK-7860
> URL: https://issues.apache.org/jira/browse/OAK-7860
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
> Reporter: Alex Deparvu
> Assignee: Alex Deparvu
> Priority: Major
> Fix For: 1.10, 1.9.10
>
>
> The PermissionEntryProviderImpl (one instance per session) is responsible for
> serving the permission entries. It can be backed by 2 different types of
> caches:
> * the {{PathEntryMapCache}} which is simply a map that eagerly loads all
> existing permissions for a session (aka. set of principals). Used only if
> there are less than 250 total entries per session (all principals combined).
> * The {{DefaultPermissionCache}} backed by the {{PermissionEntryCache}}. this
> is the fallback for the cases where the above is too expensive to use.
> In some cases, even if the number of permission entries per principal is not
> too large, a session that accumulates large numbers of group principals (I've
> seen over 700 in one internal case), will always fallback to use the
> {{PermissionEntryCache}}.
> The {{PermissionEntryCache}} is a cache that is using the principal as a key
> and the permission entries as values. It is aggregating 2 different types of
> data:
> * Existing policies: maps principals to paths with (their) existing policies
> (regular cache stuff)
> * but it also retains paths without any relevant policies for the current
> session, as empty sets [0]. being a cache keyed on the 'principal', this
> placeholder for empty policies will be set for each principal.
> I believe this second part can be responsible for a large memory footprint,
> which can also be amplified by the very large number of principals in the
> session.
> I would like to propose separating the 2 sets contained in the cache, putting
> the non-relevant paths in a dedicated map with a fixed size.
> [0]
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/security/authorization/permission/PermissionEntryCache.java#L74
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
