Hi,
By the way, the MemoryKernelImpl supports an optimization for large child
node lists, but it is disabled by default. To enable it, call:
mk.commit("/:root/head/config", "^ \"maxMemoryChildren\":" + max,
head, "");
See also org.apache.jackrabbit.mk.large.LargeNodeTest
Regards,
Thomas
On 12/15/11 10:40 PM, "Michael Dürig" <[email protected]> wrote:
>
>Hi,
>
>Starting with this revision there is some experimental support for large
>child node lists in spi2microkernel and jcr2spi (jackrabbit-mk branch in
>the sandbox). The Microkernel itself is not yet optimized for large
>child node list but AFAIK Stefan is working on this.
>
>To support large child node lists, I basically defer loading of child
>entries as much as possible. Furthermore child nodes are loaded in
>chunks of max. 512 entries at a time. Loaded child nodes are kept
>internally by the ChildNodeEntries class until cleared by the garbage
>collector. Using the garbage collector to control the child entries is
>not optimal and may lead to memory thrashing when clients fully iterate
>over large child node lists. Within the current design of the jcr2spi
>layer there does not seem to be a better solution though.
>
>Disclaimer: while the test cases pass, I did not yet test the
>effectiveness of this solution.
>
>Michael
>
>
>-------- Original Message --------
>Subject: svn commit: r1214921 -
>/jackrabbit/sandbox/jackrabbit-mk/jackrabbit-spi2microkernel/src/main/java
>/org/apache/jackrabbit/spi2microkernel/RepositoryServiceImpl.java
>Date: Thu, 15 Dec 2011 19:25:18 -0000
>From: [email protected]
>Reply-To: [email protected]
>To: [email protected]
>
>Author: mduerig
>Date: Thu Dec 15 19:25:17 2011
>New Revision: 1214921
>
>URL: http://svn.apache.org/viewvc?rev=1214921&view=rev
>Log:
>Microkernel based Jackrabbit prototype (WIP)
>batch loading of child nodes
>
>Modified:
>
>jackrabbit/sandbox/jackrabbit-mk/jackrabbit-spi2microkernel/src/main/java/
>org/apache/jackrabbit/spi2microkernel/RepositoryServiceImpl.java
>
>Modified:
>jackrabbit/sandbox/jackrabbit-mk/jackrabbit-spi2microkernel/src/main/java/
>org/apache/jackrabbit/spi2microkernel/RepositoryServiceImpl.java
>URL:
>http://svn.apache.org/viewvc/jackrabbit/sandbox/jackrabbit-mk/jackrabbit-s
>pi2microkernel/src/main/java/org/apache/jackrabbit/spi2microkernel/Reposit
>oryServiceImpl.java?rev=1214921&r1=1214920&r2=1214921&view=diff
>==========================================================================
>====
>---
>jackrabbit/sandbox/jackrabbit-mk/jackrabbit-spi2microkernel/src/main/java/
>org/apache/jackrabbit/spi2microkernel/RepositoryServiceImpl.java
>(original)
>+++
>jackrabbit/sandbox/jackrabbit-mk/jackrabbit-spi2microkernel/src/main/java/
>org/apache/jackrabbit/spi2microkernel/RepositoryServiceImpl.java
>Thu Dec 15 19:25:17 2011
>@@ -301,7 +301,7 @@ public class RepositoryServiceImpl exten
>
> @Override
> protected void checkWorkspace(String workspaceName) throws
>NoSuchWorkspaceException {
>- if (workspaceName == null || "".equals(workspaceName)) {
>+ if (workspaceName == null || workspaceName.isEmpty()) {
> workspaceName = defaultWorkspace;
> }
> else if (workspaceName.contains("/")) {
>@@ -339,7 +339,7 @@ public class RepositoryServiceImpl exten
> throw new PathNotFoundException(itemPath.toString());
> }
>
>- String json = microKernel.getNodes(mkPath, rev);
>+ String json = microKernel.getNodes(mkPath, rev, 0, 0, 0);
> return buildItemInfos(nodePath, itemPath, json,
>readFromDataStore);
> }
> catch (MicroKernelException e) {
>@@ -352,18 +352,65 @@ public class RepositoryServiceImpl exten
> public Iterator<ChildInfo> getChildInfos(SessionInfo sessionInfo,
>NodeId parentId) throws RepositoryException {
> try {
> String wspName = sessionInfo.getWorkspaceName();
>- String rev = getRevision(sessionInfo);
>- Path path = getPath(parentId, wspName, rev);
>- String mkPath = Paths.pathToString(wspName, path);
>+ final String rev = getRevision(sessionInfo);
>+ final Path path = getPath(parentId, wspName, rev);
>+ final String mkPath = Paths.pathToString(wspName, path);
>
> if (!microKernel.nodeExists(mkPath, rev)) {
> throw new PathNotFoundException(path.toString());
> }
>
>- String json = microKernel.getNodes(mkPath, rev);
>- Iterator<? extends ItemInfo> infos = buildItemInfos(path,
>path, json, readFromDataStore);
>- NodeInfo info = (NodeInfo) infos.next();
>- return info.getChildInfos();
>+ return new Iterator<ChildInfo>() {
>+ private static final int chunkSize = 512;
>+
>+ private int pos;
>+ private Iterator<ChildInfo> currentChunk;
>+
>+ {
>+ nextChunk();
>+ }
>+
>+ @Override
>+ public boolean hasNext() {
>+ if (currentChunk.hasNext()) {
>+ return true;
>+ }
>+ else {
>+ nextChunk();
>+ return currentChunk.hasNext();
>+ }
>+ }
>+
>+ @Override
>+ public ChildInfo next() {
>+ if (currentChunk.hasNext()) {
>+ return currentChunk.next();
>+ }
>+ else {
>+ nextChunk();
>+ return currentChunk.next();
>+ }
>+ }
>+
>+ @Override
>+ public void remove() {
>+ throw new UnsupportedOperationException("remove");
>+ }
>+
>+ private void nextChunk() {
>+ try {
>+ String json = microKernel.getNodes(mkPath, rev,
>0, pos, chunkSize);
>+ pos += chunkSize;
>+ Iterator<? extends ItemInfo> infos =
>buildItemInfos(path, path, json, readFromDataStore);
>+ NodeInfo info = (NodeInfo) infos.next();
>+ currentChunk = info.getChildInfos();
>+ }
>+ catch (RepositoryException e) {
>+ // fixme think of a better way to handle this
>+ log.error(e.getMessage(), e);
>+ }
>+ }
>+ };
> }
> catch (MicroKernelException e) {
> log.error(e.getMessage(), e);
>
>
>
