Hi,
I am using Jackrabbit as a version control system for files (like
CVS/SVN). I was doing some testing with large binary files and keep
getting an out of memory error.
This is most likely a configuration mistake on my side, but I can’t seem
to figure out what I’m doing wrong so hopefully someone is able to point
me into the right direction.
The file I’m trying to commit (as the jcr:data Property of an
nt:resource) is an iso image of about 700MB.
As the jcr:mimeType I used application/octet-stream in the hope that
this would prevent Lucene from indexing the file (I don’t want files
that large to be indexed). I would prefer to not completely disable
indexing. Is there a way to prevent Jackrabbit from indexing
extraordinary large properties?
As for the environment, I use Jackrabbit 1.6.0. The jackrabbit webapp is
deployed in a tomcat and the clients connect with RMI.
The heap for tomcat is set to 512m
When I try to commit this file, both client and tomcat heap space
consumption stay pretty low for most of the time. Then (I guess when all
the contents have been transferred) the tomcat heap rapidly increases
until I get an out of memory error with the stacktrace provided further
down.
My repository.xml is the default one except for a DataStorage.
I was hoping that since I did not define any TextExtractors, that
Jackrabbit would not try to index this huge binary property, but that’s
apparently where it crashes.
Thanks in advance for any pointer.
Best regards,
Johannes
Repository Config:
<!DOCTYPE Repository PUBLIC "-//The Apache Software Foundation//DTD
Jackrabbit 1.6//EN"
"http://jackrabbit.apache.org/dtd/repository-1.6.dtd";>
<Repository>
<FileSystem class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${rep.home}/repository"/>
</FileSystem>
<Security appName="Jackrabbit">
<SecurityManager
class="org.apache.jackrabbit.core.security.simple.SimpleSecurityManager"
workspaceName="security">
</SecurityManager>
<AccessManager
class="org.apache.jackrabbit.core.security.simple.SimpleAccessManager">
</AccessManager>
<LoginModule
class="org.apache.jackrabbit.core.security.simple.SimpleLoginModule">
<param name="anonymousId" value="anonymous"/>
<param name="adminId" value="admin"/>
</LoginModule>
</Security>
<Workspaces rootPath="${rep.home}/workspaces" defaultWorkspace="default"/>
<Workspace name="${wsp.name}">
<FileSystem class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${wsp.home}"/>
</FileSystem>
<PersistenceManager
class="org.apache.jackrabbit.core.persistence.bundle.DerbyPersistenceManager">
<param name="url" value="jdbc:derby:${wsp.home}/db;create=true"/>
<param name="schemaObjectPrefix" value="${wsp.name}_"/>
</PersistenceManager>
<SearchIndex class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
<param name="path" value="${wsp.home}/index"/>
<param name="extractorPoolSize" value="2"/>
<param name="supportHighlighting" value="true"/>
</SearchIndex>
</Workspace>
<Versioning rootPath="${rep.home}/version">
<FileSystem class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${rep.home}/version" />
</FileSystem>
<PersistenceManager
class="org.apache.jackrabbit.core.persistence.bundle.DerbyPersistenceManager">
<param name="url" value="jdbc:derby:${rep.home}/version/db;create=true"/>
<param name="schemaObjectPrefix" value="version_"/>
</PersistenceManager>
</Versioning>
<SearchIndex class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
<param name="path" value="${rep.home}/repository/index"/>
<param name="extractorPoolSize" value="2"/>
<param name="supportHighlighting" value="true"/>
</SearchIndex>
<DataStore class="org.apache.jackrabbit.core.data.FileDataStore">
<param name="path" value="${rep.home}/repository/datastore"/>
<param name="minRecordLength" value="100"/>
</DataStore>
</Repository>
Stacktrace:
05.02.2010 14:48:16 *WARN * LazyTextExtractorField: Exception reading
value for field: Stream closed (LazyText
ExtractorField.java, line 94)
Exception in thread "Timer-1" java.lang.IllegalStateException: this
writer hit an OutOfMemoryError; cannot com
mit
at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3353)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3408)
at
org.apache.jackrabbit.core.query.lucene.AbstractIndex.commit(AbstractIndex.java:363)
at
org.apache.jackrabbit.core.query.lucene.VolatileIndex.commit(VolatileIndex.java:141)
at
org.apache.jackrabbit.core.query.lucene.PersistentIndex.copyIndex(PersistentIndex.java:105)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex$VolatileCommit.execute(MultiIndex.java:1984)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex.executeAndLog(MultiIndex.java:1000)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex.commitVolatileIndex(MultiIndex.java:1048)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex.flush(MultiIndex.java:893)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex.checkFlush(MultiIndex.java:1164)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex.access$100(MultiIndex.java:80)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex$1.run(MultiIndex.java:317)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
05.02.2010 14:48:16 *ERROR* SearchManager: Error indexing node.
(SearchManager.java, line 490)
java.io.IOException: Java heap space
at
org.apache.jackrabbit.core.query.lucene.Util.createIOException(Util.java:114)
at
org.apache.jackrabbit.core.query.lucene.AbstractIndex.addDocuments(AbstractIndex.java:199)
at
org.apache.jackrabbit.core.query.lucene.VolatileIndex.commitPending(VolatileIndex.java:171)
at
org.apache.jackrabbit.core.query.lucene.VolatileIndex.addDocuments(VolatileIndex.java:82)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex$AddNode.execute(MultiIndex.java:1599)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex.executeAndLog(MultiIndex.java:1000)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex.update(MultiIndex.java:429)
at
org.apache.jackrabbit.core.query.lucene.SearchIndex.updateNodes(SearchIndex.java:588)
at org.apache.jackrabbit.core.SearchManager.onEvent(SearchManager.java:486)
at
org.apache.jackrabbit.core.observation.EventConsumer.consumeEvents(EventConsumer.java:244)
at
org.apache.jackrabbit.core.observation.ObservationDispatcher.dispatchEvents(ObservationDispatcher.j
ava:201)
at
org.apache.jackrabbit.core.observation.EventStateCollection.dispatch(EventStateCollection.java:464)
at
org.apache.jackrabbit.core.observation.DelegatingObservationDispatcher.dispatch(DelegatingObservati
onDispatcher.java:127)
at
org.apache.jackrabbit.core.observation.DelegatingObservationDispatcher.dispatchEvents(DelegatingObs
ervationDispatcher.java:99)
at
org.apache.jackrabbit.core.observation.EventStateCollection.dispatch(EventStateCollection.java:464)
at
org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end(SharedItemStateManager.java:760)
at
org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemStateManager.java:1115)
at
org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:351)
at
org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:326)
at
org.apache.jackrabbit.core.version.AbstractVersionManager$WriteOperation.save(AbstractVersionManage
r.java:189)
at
org.apache.jackrabbit.core.version.AbstractVersionManager.checkin(AbstractVersionManager.java:442)
at
org.apache.jackrabbit.core.version.VersionManagerImpl$2.run(VersionManagerImpl.java:290)
at
org.apache.jackrabbit.core.version.VersionManagerImpl$DynamicESCFactory.doSourced(VersionManagerImp
l.java:586)
at
org.apache.jackrabbit.core.version.VersionManagerImpl.checkin(VersionManagerImpl.java:281)
at
org.apache.jackrabbit.core.version.XAVersionManager.checkin(XAVersionManager.java:180)
at org.apache.jackrabbit.core.NodeImpl.checkin(NodeImpl.java:3367)
at org.apache.jackrabbit.core.NodeImpl.checkin(NodeImpl.java:3346)
at org.apache.jackrabbit.rmi.server.ServerNode.checkin(ServerNode.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2882)
at
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515)
at java.lang.StringBuffer.append(StringBuffer.java:306)
at
org.apache.jackrabbit.core.query.lucene.LazyTextExtractorField.stringValue(LazyTextExtractorField.j
ava:91)
at
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:109)
at
org.apache.lucene.index.DocFieldConsumersPerField.processFields(DocFieldConsumersPerField.java:36)
at
org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(DocFieldProcessorPerThread.java:
234)
at
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:765)
at
org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:743)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1917)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1895)
at
org.apache.jackrabbit.core.query.lucene.AbstractIndex$1.call(AbstractIndex.java:183)
at
org.apache.jackrabbit.core.query.lucene.DynamicPooledExecutor$1.call(DynamicPooledExecutor.java:109
)
at EDU.oswego.cs.dl.util.concurrent.FutureResult$1.run(Unknown Source)
at
EDU.oswego.cs.dl.util.concurrent.PooledExecutor$RunWhenBlocked.blockedAction(Unknown
Source)
at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.execute(Unknown Source)
at
org.apache.jackrabbit.core.query.lucene.DynamicPooledExecutor.executeAndWait(DynamicPooledExecutor.
java:113)
at
org.apache.jackrabbit.core.query.lucene.AbstractIndex.addDocuments(AbstractIndex.java:188)
at
org.apache.jackrabbit.core.query.lucene.VolatileIndex.commitPending(VolatileIndex.java:171)
at
org.apache.jackrabbit.core.query.lucene.VolatileIndex.addDocuments(VolatileIndex.java:82)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex$AddNode.execute(MultiIndex.java:1599)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex.executeAndLog(MultiIndex.java:1000)
at
org.apache.jackrabbit.core.query.lucene.MultiIndex.update(MultiIndex.java:429)
at
org.apache.jackrabbit.core.query.lucene.SearchIndex.updateNodes(SearchIndex.java:588)
at org.apache.jackrabbit.core.SearchManager.onEvent(SearchManager.java:486)
at
org.apache.jackrabbit.core.observation.EventConsumer.consumeEvents(EventConsumer.java:244)
at
org.apache.jackrabbit.core.observation.ObservationDispatcher.dispatchEvents(ObservationDispatcher.j
ava:201)
at
org.apache.jackrabbit.core.observation.EventStateCollection.dispatch(EventStateCollection.java:464)
at
org.apache.jackrabbit.core.observation.DelegatingObservationDispatcher.dispatch(DelegatingObservati
onDispatcher.java:127)
at
org.apache.jackrabbit.core.observation.DelegatingObservationDispatcher.dispatchEvents(DelegatingObs
ervationDispatcher.java:99)
at
org.apache.jackrabbit.core.observation.EventStateCollection.dispatch(EventStateCollection.java:464)
