[
https://issues.apache.org/jira/browse/OAK-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589640#comment-13589640
]
Marcel Reutegger commented on OAK-643:
--------------------------------------
Applied patch and test case in revision: 1451247
I'll still leave this issue open and have a look at how to better control the
size of the cache in KernelNodeStore.
> Very high memory usage with 6000 child nodes
> --------------------------------------------
>
> Key: OAK-643
> URL: https://issues.apache.org/jira/browse/OAK-643
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
> Reporter: Thomas Mueller
> Assignee: Marcel Reutegger
> Priority: Minor
> Attachments: OAK-643.patch
>
>
> The following test case gets slower and slower the more child nodes are added
> to a node, until (I think) it eventually runs out of memory. I have analyzed
> a heap dump, and the problem seems to be the cache in oak-core:
> {code}
> @Test
> public void testManyChildren() throws RepositoryException {
> Session session = getAdminSession();
> Node root = session.getRootNode().addNode("testRoot");
> session.save();
> int count = 100;
> for (int j = 0; j < 10; j++) {
> Node test = root.
> addNode("test" + j, "nt:folder");
> session.save();
> long time = System.currentTimeMillis();
> for (int i = 0; i < count; i++) {
> test.addNode("child" + i, "nt:folder");
> if ((i % 100) == 0) {
> session.save();
> }
> }
> System.out.println(
> count + " nodes in " +
> (System.currentTimeMillis() - time) + " ms");
> count *= 2;
> }
> }
> {code}
> Output when using the MicroKernelImpl (persisted to disk):
> {code}
> 100 nodes in 757 ms
> 200 nodes in 538 ms
> 400 nodes in 957 ms
> 800 nodes in 1630 ms
> 1600 nodes in 3245 ms
> 3200 nodes in 5389 ms
> 6400 nodes in 47581 ms
> 12800 nodes in 377216 ms
> java.lang.OutOfMemoryError: Java heap space
> {code}
> Problem Suspect 1:
> 74 MB (57.72%) in com.google.common.cache.LocalCache$Segment[]
> in a thread, see below
> Problem Suspect 2:
> 39 MB (30.76%) in org.apache.jackrabbit.mk.store.DefaultRevisionStore
> (this might be the regular node cache)
> Problem Suspect 3:
> 13 MB (10.59%) in org.h2.store.PageStore
> (16 MB is the default cache size for the H2 database,
> so this isn't a leak)
> The thread for suspect 1 is:
> {code}
> at
> java.util.LinkedHashMap.createEntry(ILjava/lang/Object;Ljava/lang/Object;I)V
> (LinkedHashMap.java:424)
> at
> java.util.LinkedHashMap.addEntry(ILjava/lang/Object;Ljava/lang/Object;I)V
> (LinkedHashMap.java:406)
> at
> java.util.HashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
> (HashMap.java:385)
> at org.apache.jackrabbit.oak.kernel.KernelNodeState.init()V
> (KernelNodeState.java:142)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeState.equals(Ljava/lang/Object;)Z
> (KernelNodeState.java:310)
> at
> org.apache.jackrabbit.oak.spi.state.AbstractNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (AbstractNodeState.java:149)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (KernelNodeState.java:286)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.merge(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeBuilder;Lorg/apache/jackrabbit/oak/spi/commit/ConflictHandler;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (MergingNodeStateDiff.java:69)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.childNodeChanged(Ljava/lang/String;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)V
> (MergingNodeStateDiff.java:90)
> at
> org.apache.jackrabbit.oak.spi.state.AbstractNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (AbstractNodeState.java:150)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (KernelNodeState.java:286)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.merge(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeBuilder;Lorg/apache/jackrabbit/oak/spi/commit/ConflictHandler;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (MergingNodeStateDiff.java:69)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.merge(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/commit/ConflictHandler;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (MergingNodeStateDiff.java:64)
> at
> org.apache.jackrabbit.oak.plugins.commit.ConflictHook.processCommit(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (ConflictHook.java:34)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (CompositeHook.java:59)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (CompositeHook.java:59)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeStoreBranch.merge(Lorg/apache/jackrabbit/oak/spi/commit/CommitHook;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (KernelNodeStoreBranch.java:128)
> at
> org.apache.jackrabbit.oak.core.RootImpl$2.run()Lorg/apache/jackrabbit/oak/api/CommitFailedException;
> (RootImpl.java:257)
> at org.apache.jackrabbit.oak.core.RootImpl$2.run()Ljava/lang/Object;
> (RootImpl.java:253)
> at
> java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;
> (Native Method)
> at
> javax.security.auth.Subject.doAs(Ljavax/security/auth/Subject;Ljava/security/PrivilegedAction;)Ljava/lang/Object;
> (Subject.java:337)
> at org.apache.jackrabbit.oak.core.RootImpl.commit()V (RootImpl.java:252)
> at org.apache.jackrabbit.oak.jcr.SessionDelegate.save()V
> (SessionDelegate.java:255)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
