[ 
https://issues.apache.org/activemq/browse/AMQ-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Tully updated AMQ-2910:
----------------------------

    Attachment: AMQ2910Test.java

Attaching a test case that can reproduce this exception. 

> NullPointerException in kahadb linked list
> ------------------------------------------
>
>                 Key: AMQ-2910
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2910
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>    Affects Versions: 5.4.0
>         Environment: java version "1.6.0_20"
> Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
> Clients connection to the broker use Active MQ 5.3.0 or 5.3.2.
>            Reporter: Gerd Behrmann
>            Priority: Critical
>         Attachments: AMQ2910Test.java
>
>
> We got the following NullPointerException in kahadb.
> 13 Sep 2010 05:53:30 (JMS) [] Caught an Exception adding a message: 
> ActiveMQObjectMessage {commandId = 111839609, responseRequired = false, 
> messageId = ID:statler.ndgf.org-42286-1282568899957-0:0:2:1:43748846, 
> originalDestination = null, originalTransactionId = null, producerId = 
> ID:statler.ndgf.org-42286-1282568899957-0:0:2:1, destination = 
> queue://cells.domain.dCacheDomain, transactionId = null, expiration = 0, 
> timestamp = 0, arrival = 0, brokerInTime = 1284350009950, brokerOutTime = 0, 
> correlationId = null, replyTo = null, persistent = false, type = null, 
> priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, 
> compressed = false, userID = null, content = 
> org.apache.activemq.util.byteseque...@1bada5d, marshalledProperties = null, 
> dataStructure = null, redeliveryCounter = 0, size = 0, properties = null, 
> readOnlyProperties = false, readOnlyBody = false, droppable = false} first to 
> FilePendingMessageCursor 
> java.lang.NullPointerException: null
>         at org.apache.kahadb.util.LinkedNode.unlink(LinkedNode.java:231) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.kahadb.util.SequenceSet.removeFirstSequence(SequenceSet.java:203) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.PageFile.allocate(PageFile.java:753) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:137) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:121) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.createEntry(PList.java:425) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:170) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:163) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.execute(Transaction.java:728) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:161) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:221)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:192)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.StoreQueueCursor.addMessageLast(StoreQueueCursor.java:96)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1601) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:707) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.send(Queue.java:646) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:365)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:494) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.scheduler.SchedulerBroker.send(SchedulerBroker.java:136)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:230) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:460)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:663) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:309)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:217)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at java.lang.Thread.run(Thread.java:619) [na:1.6.0_20]
> 13 Sep 2010 05:53:30 (JMS) [] Async error occurred: 
> java.lang.RuntimeException: java.lang.NullPointerException
> java.lang.RuntimeException: java.lang.NullPointerException
>         at 
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:228)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:192)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.StoreQueueCursor.addMessageLast(StoreQueueCursor.java:96)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1601) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:707) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.send(Queue.java:646) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:365)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:494) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.scheduler.SchedulerBroker.send(SchedulerBroker.java:136)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:230) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:460)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:663) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:309)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:217)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at java.lang.Thread.run(Thread.java:619) [na:1.6.0_20]
> Caused by: java.lang.NullPointerException: null
>         at org.apache.kahadb.util.LinkedNode.unlink(LinkedNode.java:231) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.kahadb.util.SequenceSet.removeFirstSequence(SequenceSet.java:203) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.PageFile.allocate(PageFile.java:753) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:137) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:121) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.createEntry(PList.java:425) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:170) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:163) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.execute(Transaction.java:728) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:161) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:221)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         ... 24 common frames omitted
> Looking at the code, it appears to me that the only way an NPE can be thrown 
> at that place is if list is null. Since the same method however tests for 
> that exact case, I suspect there is unsynchronized concurrent access to the 
> LinkedNode.
> A few hours later we got more failures:
> 13 Sep 2010 08:38:19 (JMS) [] Caught an Exception adding a message: 
> ActiveMQObjectMessage {commandId = 29114496, responseRequired = false, 
> messageId = ID:arctic01.csc.fi-39780-1280232016428-0:0:2:1:333447, 
> originalDestination = null, originalTransactionId = null, producerId = 
> ID:arctic01.csc.fi-39780-1280232016428-0:0:2:1, destination = 
> queue://cells.domain.dCacheDomain, transactionId = null, expiration = 0, 
> timestamp = 0, arrival = 0, brokerInTime = 1284359899636, brokerOutTime = 0, 
> correlationId = null, replyTo = null, persistent = false, type = null, 
> priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, 
> compressed = false, userID = null, content = 
> org.apache.activemq.util.byteseque...@6f28b428, marshalledProperties = null, 
> dataStructure = null, redeliveryCounter = 0, size = 0, properties = null, 
> readOnlyProperties = false, readOnlyBody = false, droppable = false} first to 
> FilePendingMessageCursor 
> java.lang.NullPointerException: null
>         at org.apache.kahadb.util.LinkedNode.isHeadNode(LinkedNode.java:64) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.util.LinkedNode.unlink(LinkedNode.java:230) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.kahadb.util.SequenceSet.removeFirstSequence(SequenceSet.java:203) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.PageFile.allocate(PageFile.java:753) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:137) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:121) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.createEntry(PList.java:425) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:170) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:163) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.execute(Transaction.java:728) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:161) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:221)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:192)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.StoreQueueCursor.addMessageLast(StoreQueueCursor.java:96)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1601) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:707) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.send(Queue.java:646) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:365)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:494) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.scheduler.SchedulerBroker.send(SchedulerBroker.java:136)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:230) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:460)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:663) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:309)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:217)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at java.lang.Thread.run(Thread.java:619) [na:1.6.0_20]
> 13 Sep 2010 08:38:19 (JMS) [] Async error occurred: 
> java.lang.RuntimeException: java.lang.NullPointerException
> java.lang.RuntimeException: java.lang.NullPointerException
>         at 
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:228)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:192)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.StoreQueueCursor.addMessageLast(StoreQueueCursor.java:96)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1601) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:707) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.send(Queue.java:646) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:365)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:494) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.scheduler.SchedulerBroker.send(SchedulerBroker.java:136)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:230) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:460)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:663) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:309)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:217)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
>  [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201) 
> [activemq-all-5.4.0.jar:5.4.0]
>         at java.lang.Thread.run(Thread.java:619) [na:1.6.0_20]
> Caused by: java.lang.NullPointerException: null
>         at org.apache.kahadb.util.LinkedNode.isHeadNode(LinkedNode.java:64) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.util.LinkedNode.unlink(LinkedNode.java:230) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.kahadb.util.SequenceSet.removeFirstSequence(SequenceSet.java:203) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.PageFile.allocate(PageFile.java:753) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:137) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:121) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.createEntry(PList.java:425) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:170) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:163) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.execute(Transaction.java:728) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:161) 
> ~[activemq-all-5.4.0.jar:5.4.0]
>         at 
> org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:221)
>  ~[activemq-all-5.4.0.jar:5.4.0]
>         ... 24 common frames omitted
> We got a number of those exceptions with the same stack trace and then 
> followed by tons of other NullPointerExceptions, likely because the list was 
> corrupted at that point.
> Not long after these stack traces message delivery began to hang. A restart 
> of the broker only helped temporarily as message delivery would hang again 
> after half a minute or so. To our big suprise moving the db directory to let 
> ActiveMQ create a new database didn't solve the problem either. I am unable 
> to tell to what extend that was caused by the above exceptions or not.
> We are happy to provide any other information you may need to debug this 
> problem.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to