Lionel Cons created AMQ-4943:
--------------------------------
Summary: Corrupted KahaDB store:
java.lang.NegativeArraySizeException
Key: AMQ-4943
URL: https://issues.apache.org/jira/browse/AMQ-4943
Project: ActiveMQ
Issue Type: Bug
Environment: activemq-5.5.1-fuse-10-16
Reporter: Lionel Cons
Priority: Critical
One of our brokers went crazy and logged _many_ exceptions looking like this:
2013-12-19 12:27:09,007 [BrokerService[foobar] Task-13] ERROR
AbstractStoreCursor -
org.apache.activemq.broker.region.cursors.QueueStorePrefetch@654e3615:Consumer.prod.whatever,batchResetNeeded=false,storeHasMessages=true,size=2461725,cacheEnabled=false,maxBatchSize:200
- Failed to fill batch
java.lang.RuntimeException: java.io.IOException: Invalid location: 43153:28, :
java.lang.NegativeArraySizeException
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:277)
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:110)
at
org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:157)
at
org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1746)
at
org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:1962)
at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1470)
at
org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
at
org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Invalid location: 43153:28, :
java.lang.NegativeArraySizeException
at
org.apache.kahadb.journal.DataFileAccessor.readRecord(DataFileAccessor.java:94)
at org.apache.kahadb.journal.Journal.read(Journal.java:601)
at
org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:908)
at
org.apache.activemq.store.kahadb.KahaDBStore.loadMessage(KahaDBStore.java:1024)
at
org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore$4.execute(KahaDBStore.java:552)
at org.apache.kahadb.page.Transaction.execute(Transaction.java:771)
at
org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.recoverNextMessages(KahaDBStore.java:541)
at
org.apache.activemq.store.ProxyMessageStore.recoverNextMessages(ProxyMessageStore.java:88)
at
org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:97)
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:274)
... 10 more
2013-12-19 12:27:09,007 [BrokerService[foobar] Task-13] ERROR Queue - Failed to
page in more queue messages
java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException:
Invalid location: 43153:28, : java.lang.NegativeArraySizeException
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:113)
at
org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:157)
at
org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1746)
at
org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:1962)
at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1470)
at
org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
at
org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: java.io.IOException: Invalid location:
43153:28, : java.lang.NegativeArraySizeException
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:277)
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:110)
... 9 more
Caused by: java.io.IOException: Invalid location: 43153:28, :
java.lang.NegativeArraySizeException
at
org.apache.kahadb.journal.DataFileAccessor.readRecord(DataFileAccessor.java:94)
at org.apache.kahadb.journal.Journal.read(Journal.java:601)
at
org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:908)
at
org.apache.activemq.store.kahadb.KahaDBStore.loadMessage(KahaDBStore.java:1024)
at
org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore$4.execute(KahaDBStore.java:552)
at org.apache.kahadb.page.Transaction.execute(Transaction.java:771)
at
org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.recoverNextMessages(KahaDBStore.java:541)
at
org.apache.activemq.store.ProxyMessageStore.recoverNextMessages(ProxyMessageStore.java:88)
at
org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:97)
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:274)
A restart did not hep, the KahaDB stores is corrupted :-(
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
