[
https://issues.apache.org/jira/browse/JCR-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek updated JCR-3873:
---------------------------------------
Description:
An NPE in OAK-2811 lead to the data store not being closed normally on
shutdown, in turn leaving the {{async-pending-uploads.ser}} file corrupted (on
this system, binaries were constantly uploaded, so likely the queue to upload
it to S3 in this case was still full) and then on the next startup, the
CachingDataStore failed to start up, preventing the repository and the system
from starting:
{noformat}
24.04.2015 13:21:28.602 *ERROR* [FelixStartLevel]
org.apache.jackrabbit.oak-core
[org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore(79)] The activate
method has thrown an exception (javax.jcr.RepositoryException:
java.io.EOFException)
javax.jcr.RepositoryException: java.io.EOFException
at
org.apache.jackrabbit.core.data.CachingDataStore.init(CachingDataStore.java:337)
at
org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
at
org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
at
org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
at
org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:499)
at
org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:295)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:302)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:113)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:832)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:799)
at
org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:724)
at
org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:399)
at
org.apache.felix.scr.impl.config.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:669)
at
org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:184)
at
org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:332)
at org.apache.felix.scr.impl.Activator.access$000(Activator.java:49)
at
org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:257)
at
org.apache.felix.utils.extender.AbstractExtender.createExtension(AbstractExtender.java:259)
at
org.apache.felix.utils.extender.AbstractExtender.modifiedBundle(AbstractExtender.java:232)
at
org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:482)
at
org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:415)
at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)
at
org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:444)
at
org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:913)
at
org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:834)
at
org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:516)
at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4531)
at org.apache.felix.framework.Felix.startBundle(Felix.java:2169)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1368)
at
org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.EOFException: null
at
java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325)
at
java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2794)
at
java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:801)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299)
at
org.apache.jackrabbit.core.data.AsyncUploadCache.deserializeAsyncUploadMap(AsyncUploadCache.java:311)
at
org.apache.jackrabbit.core.data.AsyncUploadCache.init(AsyncUploadCache.java:243)
at
org.apache.jackrabbit.core.data.CachingDataStore.init(CachingDataStore.java:264)
... 35 common frames omitted
{noformat}
We had to manually remove the file so that the system would start again. AFAIU,
this meant some data loss.
was:
An NPE in OAK-2811 lead to the data store not being closed normally on
shutdown, in turn leaving the {{async-pending-uploads.ser}} file corrupted (on
this system, binaries were constantly uploaded, so likely the queue to upload
it to S3 in this case was still full) and then on the next startup, the
CachingDataStore failed to start up, preventing the repository and the system
from starting:
{noformat}
24.04.2015 13:21:28.602 *ERROR* [FelixStartLevel]
org.apache.jackrabbit.oak-core
[org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore(79)] The activate
method has thrown an exception (javax.jcr.RepositoryException:
java.io.EOFException)
javax.jcr.RepositoryException: java.io.EOFException
at
org.apache.jackrabbit.core.data.CachingDataStore.init(CachingDataStore.java:337)
at
org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
at
org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
at
org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
at
org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:499)
at
org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:295)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:302)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:113)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:832)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:799)
at
org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:724)
at
org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:399)
at
org.apache.felix.scr.impl.config.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:669)
at
org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:184)
at
org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:332)
at org.apache.felix.scr.impl.Activator.access$000(Activator.java:49)
at
org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:257)
at
org.apache.felix.utils.extender.AbstractExtender.createExtension(AbstractExtender.java:259)
at
org.apache.felix.utils.extender.AbstractExtender.modifiedBundle(AbstractExtender.java:232)
at
org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:482)
at
org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:415)
at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)
at
org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:444)
at
org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:913)
at
org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:834)
at
org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:516)
at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4531)
at org.apache.felix.framework.Felix.startBundle(Felix.java:2169)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1368)
at
org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.EOFException: null
at
java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325)
at
java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2794)
at
java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:801)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299)
at
org.apache.jackrabbit.core.data.AsyncUploadCache.deserializeAsyncUploadMap(AsyncUploadCache.java:311)
at
org.apache.jackrabbit.core.data.AsyncUploadCache.init(AsyncUploadCache.java:243)
at
org.apache.jackrabbit.core.data.CachingDataStore.init(CachingDataStore.java:264)
... 35 common frames omitted
{noformat}
> CachingDataStore not safe against crashes, corrupted uploads file will
> prevent system startup
> ---------------------------------------------------------------------------------------------
>
> Key: JCR-3873
> URL: https://issues.apache.org/jira/browse/JCR-3873
> Project: Jackrabbit Content Repository
> Issue Type: Bug
> Components: jackrabbit-data
> Affects Versions: 2.9.1
> Reporter: Alexander Klimetschek
>
> An NPE in OAK-2811 lead to the data store not being closed normally on
> shutdown, in turn leaving the {{async-pending-uploads.ser}} file corrupted
> (on this system, binaries were constantly uploaded, so likely the queue to
> upload it to S3 in this case was still full) and then on the next startup,
> the CachingDataStore failed to start up, preventing the repository and the
> system from starting:
> {noformat}
> 24.04.2015 13:21:28.602 *ERROR* [FelixStartLevel]
> org.apache.jackrabbit.oak-core
> [org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore(79)] The
> activate method has thrown an exception (javax.jcr.RepositoryException:
> java.io.EOFException)
> javax.jcr.RepositoryException: java.io.EOFException
> at
> org.apache.jackrabbit.core.data.CachingDataStore.init(CachingDataStore.java:337)
> at
> org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:61)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
> at
> org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
> at
> org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
> at
> org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:499)
> at
> org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:295)
> at
> org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:302)
> at
> org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:113)
> at
> org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:832)
> at
> org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:799)
> at
> org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:724)
> at
> org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:399)
> at
> org.apache.felix.scr.impl.config.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:669)
> at
> org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:184)
> at
> org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:332)
> at org.apache.felix.scr.impl.Activator.access$000(Activator.java:49)
> at
> org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:257)
> at
> org.apache.felix.utils.extender.AbstractExtender.createExtension(AbstractExtender.java:259)
> at
> org.apache.felix.utils.extender.AbstractExtender.modifiedBundle(AbstractExtender.java:232)
> at
> org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:482)
> at
> org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:415)
> at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)
> at
> org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:444)
> at
> org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:913)
> at
> org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:834)
> at
> org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:516)
> at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4531)
> at org.apache.felix.framework.Felix.startBundle(Felix.java:2169)
> at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1368)
> at
> org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.EOFException: null
> at
> java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325)
> at
> java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2794)
> at
> java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:801)
> at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299)
> at
> org.apache.jackrabbit.core.data.AsyncUploadCache.deserializeAsyncUploadMap(AsyncUploadCache.java:311)
> at
> org.apache.jackrabbit.core.data.AsyncUploadCache.init(AsyncUploadCache.java:243)
> at
> org.apache.jackrabbit.core.data.CachingDataStore.init(CachingDataStore.java:264)
> ... 35 common frames omitted
> {noformat}
> We had to manually remove the file so that the system would start again.
> AFAIU, this meant some data loss.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
