[
https://issues.apache.org/jira/browse/AMQ-5081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Håkan Dahl updated AMQ-5081:
----------------------------
Description:
Seems related to APLO-284.
Problem occurs occasionally after restart of broker when using LevelDB.
The broker starts to log very intensively (doesn't seem to stop):
---
jvm 1 | WARN | Invalid log position: 79672162
jvm 1 | WARN | Invalid log position: 79673204
jvm 1 | WARN | Invalid log position: 79674246
jvm 1 | WARN | Invalid log position: 79683816
jvm 1 | WARN | Invalid log position: 79686990
jvm 1 | WARN | Invalid log position: 79690164
---
After a second restart the logging has stopped but the broker is constantly
restarting itself due to a LevelDB problem (see attached logfile), had to
remove the leveldb-dir from disk to get the broker to run normally again (see
attached leveldb-store containing the problem):
2014-03-04 13:59:38,443 | INFO | Stopping BrokerService[localhost] due to
exception, java.io.IOException |
org.apache.activemq.util.DefaultIOExceptionHandler | LevelDB IOException
handler.
java.io.IOException
at
org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:39)
at
org.apache.activemq.leveldb.LevelDBClient.might_fail(LevelDBClient.scala:543)
at
org.apache.activemq.leveldb.LevelDBClient.might_fail_using_index(LevelDBClient.scala:974)
at
org.apache.activemq.leveldb.LevelDBClient.collectionCursor(LevelDBClient.scala:1270)
at
org.apache.activemq.leveldb.LevelDBClient.queueCursor(LevelDBClient.scala:1194)
at
org.apache.activemq.leveldb.DBManager.cursorMessages(DBManager.scala:708)
at
org.apache.activemq.leveldb.LevelDBStore$LevelDBMessageStore.recoverNextMessages(LevelDBStore.scala:741)
at
org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:106)
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:258)
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:108)
at
org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:157)
at
org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1875)
at
org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2086)
at org.apache.activemq.broker.region.Queue.doBrowse(Queue.java:1096)
at
org.apache.activemq.broker.region.Queue.expireMessages(Queue.java:905)
at org.apache.activemq.broker.region.Queue.access$100(Queue.java:79)
at org.apache.activemq.broker.region.Queue$2.run(Queue.java:120)
at
org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
Caused by: java.lang.NullPointerException
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:1198)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:1194)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$12.apply(LevelDBClient.scala:1272)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$12.apply(LevelDBClient.scala:1271)
at
org.apache.activemq.leveldb.LevelDBClient$RichDB.check$4(LevelDBClient.scala:315)
was:
Seems related to APLO-284.
Problem occurs occasionally after restart of broker when using LevelDB.
The broker starts to log very intensively (doesn't seem to stop):
---
jvm 1 | WARN | Invalid log position: 79672162
jvm 1 | WARN | Invalid log position: 79673204
jvm 1 | WARN | Invalid log position: 79674246
jvm 1 | WARN | Invalid log position: 79683816
jvm 1 | WARN | Invalid log position: 79686990
jvm 1 | WARN | Invalid log position: 79690164
---
After a second restart the logging has stopped but the broker is constantly
restarting itself due to a LevelDB problem (see attached logfile):
2014-03-04 13:59:38,443 | INFO | Stopping BrokerService[localhost] due to
exception, java.io.IOException |
org.apache.activemq.util.DefaultIOExceptionHandler | LevelDB IOException
handler.
java.io.IOException
at
org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:39)
at
org.apache.activemq.leveldb.LevelDBClient.might_fail(LevelDBClient.scala:543)
at
org.apache.activemq.leveldb.LevelDBClient.might_fail_using_index(LevelDBClient.scala:974)
at
org.apache.activemq.leveldb.LevelDBClient.collectionCursor(LevelDBClient.scala:1270)
at
org.apache.activemq.leveldb.LevelDBClient.queueCursor(LevelDBClient.scala:1194)
at
org.apache.activemq.leveldb.DBManager.cursorMessages(DBManager.scala:708)
at
org.apache.activemq.leveldb.LevelDBStore$LevelDBMessageStore.recoverNextMessages(LevelDBStore.scala:741)
at
org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:106)
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:258)
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:108)
at
org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:157)
at
org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1875)
at
org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2086)
at org.apache.activemq.broker.region.Queue.doBrowse(Queue.java:1096)
at
org.apache.activemq.broker.region.Queue.expireMessages(Queue.java:905)
at org.apache.activemq.broker.region.Queue.access$100(Queue.java:79)
at org.apache.activemq.broker.region.Queue$2.run(Queue.java:120)
at
org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
at java.util.TimerThread.mainLoop(Unknown Source)
at java.util.TimerThread.run(Unknown Source)
Caused by: java.lang.NullPointerException
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:1198)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:1194)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$12.apply(LevelDBClient.scala:1272)
at
org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$12.apply(LevelDBClient.scala:1271)
at
org.apache.activemq.leveldb.LevelDBClient$RichDB.check$4(LevelDBClient.scala:315)
> LevelDB logging "WARN | Invalid log position" after restart
> -----------------------------------------------------------
>
> Key: AMQ-5081
> URL: https://issues.apache.org/jira/browse/AMQ-5081
> Project: ActiveMQ
> Issue Type: Bug
> Components: activemq-leveldb-store
> Affects Versions: 5.9.0
> Environment: Win7/64bit
> Reporter: Håkan Dahl
> Attachments: activemq.log, leveldb.zip, wrapper.log
>
>
> Seems related to APLO-284.
> Problem occurs occasionally after restart of broker when using LevelDB.
> The broker starts to log very intensively (doesn't seem to stop):
> ---
> jvm 1 | WARN | Invalid log position: 79672162
> jvm 1 | WARN | Invalid log position: 79673204
> jvm 1 | WARN | Invalid log position: 79674246
> jvm 1 | WARN | Invalid log position: 79683816
> jvm 1 | WARN | Invalid log position: 79686990
> jvm 1 | WARN | Invalid log position: 79690164
> ---
> After a second restart the logging has stopped but the broker is constantly
> restarting itself due to a LevelDB problem (see attached logfile), had to
> remove the leveldb-dir from disk to get the broker to run normally again (see
> attached leveldb-store containing the problem):
> 2014-03-04 13:59:38,443 | INFO | Stopping BrokerService[localhost] due to
> exception, java.io.IOException |
> org.apache.activemq.util.DefaultIOExceptionHandler | LevelDB IOException
> handler.
> java.io.IOException
> at
> org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:39)
> at
> org.apache.activemq.leveldb.LevelDBClient.might_fail(LevelDBClient.scala:543)
> at
> org.apache.activemq.leveldb.LevelDBClient.might_fail_using_index(LevelDBClient.scala:974)
> at
> org.apache.activemq.leveldb.LevelDBClient.collectionCursor(LevelDBClient.scala:1270)
> at
> org.apache.activemq.leveldb.LevelDBClient.queueCursor(LevelDBClient.scala:1194)
> at
> org.apache.activemq.leveldb.DBManager.cursorMessages(DBManager.scala:708)
> at
> org.apache.activemq.leveldb.LevelDBStore$LevelDBMessageStore.recoverNextMessages(LevelDBStore.scala:741)
> at
> org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:106)
> at
> org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:258)
> at
> org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:108)
> at
> org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:157)
> at
> org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1875)
> at
> org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2086)
> at org.apache.activemq.broker.region.Queue.doBrowse(Queue.java:1096)
> at
> org.apache.activemq.broker.region.Queue.expireMessages(Queue.java:905)
> at org.apache.activemq.broker.region.Queue.access$100(Queue.java:79)
> at org.apache.activemq.broker.region.Queue$2.run(Queue.java:120)
> at
> org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
> at java.util.TimerThread.mainLoop(Unknown Source)
> at java.util.TimerThread.run(Unknown Source)
> Caused by: java.lang.NullPointerException
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:1198)
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$queueCursor$1.apply(LevelDBClient.scala:1194)
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$12.apply(LevelDBClient.scala:1272)
> at
> org.apache.activemq.leveldb.LevelDBClient$$anonfun$collectionCursor$1$$anonfun$apply$mcV$sp$12.apply(LevelDBClient.scala:1271)
> at
> org.apache.activemq.leveldb.LevelDBClient$RichDB.check$4(LevelDBClient.scala:315)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
