[
https://issues.apache.org/jira/browse/OAK-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582230#comment-13582230
]
Thomas Mueller commented on OAK-643:
------------------------------------
Thanks Marcel, I linked OAK-373.
I guess we need to change the cache to account for the memory usage of each
entry. To do that, each cache entry (KernelNodeState objects) would need to be
internally immutable, or the cache would need to be informed when the size of
an entry changes. I would prefer immutable objects, but I'm not sure if that's
feasible.
Jukka, as far as I remember you once said that entries in this cache are not
internally immutable at the moment? I don't currently see that in the code, did
this change recently?
> The key here is AbstractNodeState.compareAgainstBaseState
I don't currently see how this is related to this (memory usage) problem. I
think you talk about a different problem, about a performance problem, right?
> Very high memory usage with 6000 child nodes
> --------------------------------------------
>
> Key: OAK-643
> URL: https://issues.apache.org/jira/browse/OAK-643
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
> Reporter: Thomas Mueller
> Priority: Minor
>
> The following test case gets slower and slower the more child nodes are added
> to a node, until (I think) it eventually runs out of memory. I have analyzed
> a heap dump, and the problem seems to be the cache in oak-core:
> {code}
> @Test
> public void testManyChildren() throws RepositoryException {
> Session session = getAdminSession();
> Node root = session.getRootNode().addNode("testRoot");
> session.save();
> int count = 100;
> for (int j = 0; j < 10; j++) {
> Node test = root.
> addNode("test" + j, "nt:folder");
> session.save();
> long time = System.currentTimeMillis();
> for (int i = 0; i < count; i++) {
> test.addNode("child" + i, "nt:folder");
> if ((i % 100) == 0) {
> session.save();
> }
> }
> System.out.println(
> count + " nodes in " +
> (System.currentTimeMillis() - time) + " ms");
> count *= 2;
> }
> }
> {code}
> Output when using the MicroKernelImpl (persisted to disk):
> {code}
> 100 nodes in 757 ms
> 200 nodes in 538 ms
> 400 nodes in 957 ms
> 800 nodes in 1630 ms
> 1600 nodes in 3245 ms
> 3200 nodes in 5389 ms
> 6400 nodes in 47581 ms
> 12800 nodes in 377216 ms
> java.lang.OutOfMemoryError: Java heap space
> {code}
> Problem Suspect 1:
> 74 MB (57.72%) in com.google.common.cache.LocalCache$Segment[]
> in a thread, see below
> Problem Suspect 2:
> 39 MB (30.76%) in org.apache.jackrabbit.mk.store.DefaultRevisionStore
> (this might be the regular node cache)
> Problem Suspect 3:
> 13 MB (10.59%) in org.h2.store.PageStore
> (16 MB is the default cache size for the H2 database,
> so this isn't a leak)
> The thread for suspect 1 is:
> {code}
> at
> java.util.LinkedHashMap.createEntry(ILjava/lang/Object;Ljava/lang/Object;I)V
> (LinkedHashMap.java:424)
> at
> java.util.LinkedHashMap.addEntry(ILjava/lang/Object;Ljava/lang/Object;I)V
> (LinkedHashMap.java:406)
> at
> java.util.HashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
> (HashMap.java:385)
> at org.apache.jackrabbit.oak.kernel.KernelNodeState.init()V
> (KernelNodeState.java:142)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeState.equals(Ljava/lang/Object;)Z
> (KernelNodeState.java:310)
> at
> org.apache.jackrabbit.oak.spi.state.AbstractNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (AbstractNodeState.java:149)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (KernelNodeState.java:286)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.merge(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeBuilder;Lorg/apache/jackrabbit/oak/spi/commit/ConflictHandler;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (MergingNodeStateDiff.java:69)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.childNodeChanged(Ljava/lang/String;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)V
> (MergingNodeStateDiff.java:90)
> at
> org.apache.jackrabbit.oak.spi.state.AbstractNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (AbstractNodeState.java:150)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (KernelNodeState.java:286)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.merge(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeBuilder;Lorg/apache/jackrabbit/oak/spi/commit/ConflictHandler;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (MergingNodeStateDiff.java:69)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.merge(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/commit/ConflictHandler;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (MergingNodeStateDiff.java:64)
> at
> org.apache.jackrabbit.oak.plugins.commit.ConflictHook.processCommit(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (ConflictHook.java:34)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (CompositeHook.java:59)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (CompositeHook.java:59)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeStoreBranch.merge(Lorg/apache/jackrabbit/oak/spi/commit/CommitHook;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (KernelNodeStoreBranch.java:128)
> at
> org.apache.jackrabbit.oak.core.RootImpl$2.run()Lorg/apache/jackrabbit/oak/api/CommitFailedException;
> (RootImpl.java:257)
> at org.apache.jackrabbit.oak.core.RootImpl$2.run()Ljava/lang/Object;
> (RootImpl.java:253)
> at
> java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;
> (Native Method)
> at
> javax.security.auth.Subject.doAs(Ljavax/security/auth/Subject;Ljava/security/PrivilegedAction;)Ljava/lang/Object;
> (Subject.java:337)
> at org.apache.jackrabbit.oak.core.RootImpl.commit()V (RootImpl.java:252)
> at org.apache.jackrabbit.oak.jcr.SessionDelegate.save()V
> (SessionDelegate.java:255)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
