[
https://issues.apache.org/jira/browse/OAK-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13582254#comment-13582254
]
Marcel Reutegger commented on OAK-643:
--------------------------------------
KernelNodeStates are lazy initialized when they are already in the cache. See
KernelNodeState.init(). The Guava cache is designed to weigh the cached item
when it is added to the cache.
bq. The key here is AbstractNodeState.compareAgainstBaseState
Jukka is probably refering to the fact that adding a child node to a parent
with many children forces loading all of the children later to find out, which
one was added. Delegating this to the MicroKernel.diff() might be more efficent
in this case.
> Very high memory usage with 6000 child nodes
> --------------------------------------------
>
> Key: OAK-643
> URL: https://issues.apache.org/jira/browse/OAK-643
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
> Reporter: Thomas Mueller
> Priority: Minor
>
> The following test case gets slower and slower the more child nodes are added
> to a node, until (I think) it eventually runs out of memory. I have analyzed
> a heap dump, and the problem seems to be the cache in oak-core:
> {code}
> @Test
> public void testManyChildren() throws RepositoryException {
> Session session = getAdminSession();
> Node root = session.getRootNode().addNode("testRoot");
> session.save();
> int count = 100;
> for (int j = 0; j < 10; j++) {
> Node test = root.
> addNode("test" + j, "nt:folder");
> session.save();
> long time = System.currentTimeMillis();
> for (int i = 0; i < count; i++) {
> test.addNode("child" + i, "nt:folder");
> if ((i % 100) == 0) {
> session.save();
> }
> }
> System.out.println(
> count + " nodes in " +
> (System.currentTimeMillis() - time) + " ms");
> count *= 2;
> }
> }
> {code}
> Output when using the MicroKernelImpl (persisted to disk):
> {code}
> 100 nodes in 757 ms
> 200 nodes in 538 ms
> 400 nodes in 957 ms
> 800 nodes in 1630 ms
> 1600 nodes in 3245 ms
> 3200 nodes in 5389 ms
> 6400 nodes in 47581 ms
> 12800 nodes in 377216 ms
> java.lang.OutOfMemoryError: Java heap space
> {code}
> Problem Suspect 1:
> 74 MB (57.72%) in com.google.common.cache.LocalCache$Segment[]
> in a thread, see below
> Problem Suspect 2:
> 39 MB (30.76%) in org.apache.jackrabbit.mk.store.DefaultRevisionStore
> (this might be the regular node cache)
> Problem Suspect 3:
> 13 MB (10.59%) in org.h2.store.PageStore
> (16 MB is the default cache size for the H2 database,
> so this isn't a leak)
> The thread for suspect 1 is:
> {code}
> at
> java.util.LinkedHashMap.createEntry(ILjava/lang/Object;Ljava/lang/Object;I)V
> (LinkedHashMap.java:424)
> at
> java.util.LinkedHashMap.addEntry(ILjava/lang/Object;Ljava/lang/Object;I)V
> (LinkedHashMap.java:406)
> at
> java.util.HashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
> (HashMap.java:385)
> at org.apache.jackrabbit.oak.kernel.KernelNodeState.init()V
> (KernelNodeState.java:142)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeState.equals(Ljava/lang/Object;)Z
> (KernelNodeState.java:310)
> at
> org.apache.jackrabbit.oak.spi.state.AbstractNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (AbstractNodeState.java:149)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (KernelNodeState.java:286)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.merge(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeBuilder;Lorg/apache/jackrabbit/oak/spi/commit/ConflictHandler;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (MergingNodeStateDiff.java:69)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.childNodeChanged(Ljava/lang/String;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)V
> (MergingNodeStateDiff.java:90)
> at
> org.apache.jackrabbit.oak.spi.state.AbstractNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (AbstractNodeState.java:150)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeState.compareAgainstBaseState(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeStateDiff;)V
> (KernelNodeState.java:286)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.merge(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeBuilder;Lorg/apache/jackrabbit/oak/spi/commit/ConflictHandler;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (MergingNodeStateDiff.java:69)
> at
> org.apache.jackrabbit.oak.plugins.commit.MergingNodeStateDiff.merge(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/commit/ConflictHandler;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (MergingNodeStateDiff.java:64)
> at
> org.apache.jackrabbit.oak.plugins.commit.ConflictHook.processCommit(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (ConflictHook.java:34)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (CompositeHook.java:59)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeHook.processCommit(Lorg/apache/jackrabbit/oak/spi/state/NodeState;Lorg/apache/jackrabbit/oak/spi/state/NodeState;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (CompositeHook.java:59)
> at
> org.apache.jackrabbit.oak.kernel.KernelNodeStoreBranch.merge(Lorg/apache/jackrabbit/oak/spi/commit/CommitHook;)Lorg/apache/jackrabbit/oak/spi/state/NodeState;
> (KernelNodeStoreBranch.java:128)
> at
> org.apache.jackrabbit.oak.core.RootImpl$2.run()Lorg/apache/jackrabbit/oak/api/CommitFailedException;
> (RootImpl.java:257)
> at org.apache.jackrabbit.oak.core.RootImpl$2.run()Ljava/lang/Object;
> (RootImpl.java:253)
> at
> java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;
> (Native Method)
> at
> javax.security.auth.Subject.doAs(Ljavax/security/auth/Subject;Ljava/security/PrivilegedAction;)Ljava/lang/Object;
> (Subject.java:337)
> at org.apache.jackrabbit.oak.core.RootImpl.commit()V (RootImpl.java:252)
> at org.apache.jackrabbit.oak.jcr.SessionDelegate.save()V
> (SessionDelegate.java:255)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
