Deadlock between VMTransport threads accessing AMQStore
-------------------------------------------------------
Key: AMQ-1564
URL: https://issues.apache.org/activemq/browse/AMQ-1564
Project: ActiveMQ
Issue Type: Bug
Components: Message Store
Affects Versions: 5.1.0
Environment: Linux
Reporter: Rainer Klute
Fix For: 5.1.0
A some time and quite a lot of ActiveMQ patches I dared to run my favorite
Reliability JUnit test (see AMQ-1490) on the current ActiveMQ snapshot
(revision 614484). Unfortunately already the first test case failed with a
deadlock after processing so many messages.
Here are the stack traces of what ''jconsole'' considered the threads involved
in the deadlock. They obviously have to do with the AMQStore:
<pre>
Name: VMTransport
State: BLOCKED on [EMAIL PROTECTED] owned by: VMTransport
Total blocked: 341 Total waited: 3.750
Stack trace:
org.apache.activemq.store.kahadaptor.KahaTopicReferenceStore.acknowledgeReference(KahaTopicReferenceStore.java:127)
org.apache.activemq.store.amq.AMQTopicMessageStore.acknowledge(AMQTopicMessageStore.java:154)
org.apache.activemq.store.amq.AMQTopicMessageStore$1.afterCommit(AMQTopicMessageStore.java:113)
- locked [EMAIL PROTECTED]
org.apache.activemq.transaction.Transaction.fireAfterCommit(Transaction.java:86)
org.apache.activemq.transaction.LocalTransaction.commit(LocalTransaction.java:68)
org.apache.activemq.broker.TransactionBroker.commitTransaction(TransactionBroker.java:170)
org.apache.activemq.broker.BrokerFilter.commitTransaction(BrokerFilter.java:93)
org.apache.activemq.broker.BrokerFilter.commitTransaction(BrokerFilter.java:93)
org.apache.activemq.broker.MutableBrokerFilter.commitTransaction(MutableBrokerFilter.java:103)
org.apache.activemq.broker.TransportConnection.processCommitTransactionOnePhase(TransportConnection.java:396)
org.apache.activemq.command.TransactionInfo.visit(TransactionInfo.java:100)
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:280)
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:177)
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:100)
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:67)
org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:202)
org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:118)
org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:42)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
java.lang.Thread.run(Thread.java:619)
</pre>
And the second one:
<pre>
Name: VMTransport
State: BLOCKED on [EMAIL PROTECTED] owned by: VMTransport
Total blocked: 294 Total waited: 5.570
Stack trace:
org.apache.activemq.store.amq.AMQMessageStore.getLocation(AMQMessageStore.java:406)
org.apache.activemq.store.amq.AMQMessageStore.getMessage(AMQMessageStore.java:390)
org.apache.activemq.store.amq.RecoveryListenerAdapter.recoverMessageReference(RecoveryListenerAdapter.java:54)
org.apache.activemq.store.kahadaptor.KahaReferenceStore.recoverReference(KahaReferenceStore.java:70)
org.apache.activemq.store.kahadaptor.KahaTopicReferenceStore.recoverNextMessages(KahaTopicReferenceStore.java:258)
- locked [EMAIL PROTECTED]
org.apache.activemq.store.amq.AMQTopicMessageStore.recoverNextMessages(AMQTopicMessageStore.java:64)
org.apache.activemq.broker.region.cursors.TopicStorePrefetch.fillBatch(TopicStorePrefetch.java:215)
- locked [EMAIL PROTECTED]
org.apache.activemq.broker.region.cursors.TopicStorePrefetch.safeFillBatch(TopicStorePrefetch.java:200)
- locked [EMAIL PROTECTED]
org.apache.activemq.broker.region.cursors.TopicStorePrefetch.isEmpty(TopicStorePrefetch.java:96)
- locked [EMAIL PROTECTED]
org.apache.activemq.broker.region.cursors.StoreDurableSubscriberCursor.getNextCursor(StoreDurableSubscriberCursor.java:307)
- locked [EMAIL PROTECTED]
org.apache.activemq.broker.region.cursors.StoreDurableSubscriberCursor.hasNext(StoreDurableSubscriberCursor.java:190)
- locked [EMAIL PROTECTED]
org.apache.activemq.broker.region.PrefetchSubscription.dispatchPending(PrefetchSubscription.java:481)
- locked [EMAIL PROTECTED]
org.apache.activemq.broker.region.PrefetchSubscription.add(PrefetchSubscription.java:146)
org.apache.activemq.broker.region.DurableTopicSubscription.add(DurableTopicSubscription.java:178)
org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:48)
org.apache.activemq.broker.region.Topic.dispatch(Topic.java:607)
org.apache.activemq.broker.region.Topic$4.afterCommit(Topic.java:424)
org.apache.activemq.transaction.Transaction.fireAfterCommit(Transaction.java:86)
org.apache.activemq.transaction.LocalTransaction.commit(LocalTransaction.java:68)
org.apache.activemq.broker.TransactionBroker.commitTransaction(TransactionBroker.java:170)
org.apache.activemq.broker.BrokerFilter.commitTransaction(BrokerFilter.java:93)
org.apache.activemq.broker.BrokerFilter.commitTransaction(BrokerFilter.java:93)
org.apache.activemq.broker.MutableBrokerFilter.commitTransaction(MutableBrokerFilter.java:103)
org.apache.activemq.broker.TransportConnection.processCommitTransactionOnePhase(TransportConnection.java:396)
org.apache.activemq.command.TransactionInfo.visit(TransactionInfo.java:100)
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:280)
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:177)
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:100)
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:67)
org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:202)
org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:118)
org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:42)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
java.lang.Thread.run(Thread.java:619)
</pre>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
