[
https://issues.apache.org/jira/browse/AMQ-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313450#comment-17313450
]
Andrew Weinstein commented on AMQ-6660:
---------------------------------------
Please re-open this ticket to investigate since we're seeing this deadlock
again even after upgrading to 5.16.1:
{noformat}
"ActiveMQ Connection Executor:
tcp://********":
waiting to lock monitor 0x00007fb9ec008898 (object 0x000000008512c448, a
java.lang.Object),
which is held by "DefaultMessageListenerContainer-502"
"DefaultMessageListenerContainer-502":
waiting to lock monitor 0x00007fba440325e8 (object 0x0000000085048718, a
org.apache.activemq.ActiveMQConnection),
which is held by "ActiveMQ Connection Executor: tcp://********"
"ActiveMQ Connection Executor:
tcp://********":
at
org.apache.activemq.SimplePriorityMessageDispatchChannel.close(SimplePriorityMessageDispatchChannel.java:166)
- waiting to lock <0x000000008512c448> (a java.lang.Object)
at
org.apache.activemq.ActiveMQMessageConsumer.dispose(ActiveMQMessageConsumer.java:859)
at org.apache.activemq.ActiveMQSession.dispose(ActiveMQSession.java:729)
- locked <0x000000008506fd50> (a org.apache.activemq.ActiveMQSession)
at
org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:671)
- locked <0x0000000085048718> (a org.apache.activemq.ActiveMQConnection)
at
org.springframework.jms.connection.SingleConnectionFactory.closeConnection(SingleConnectionFactory.java:501)
at
org.springframework.jms.connection.SingleConnectionFactory.resetConnection(SingleConnectionFactory.java:389)
- locked <0x000000008145d510> (a java.lang.Object)
at
org.springframework.jms.connection.CachingConnectionFactory.resetConnection(CachingConnectionFactory.java:205)
at
org.springframework.jms.connection.SingleConnectionFactory.onException(SingleConnectionFactory.java:367)
at
org.springframework.jms.connection.SingleConnectionFactory$AggregatedExceptionListener.onException(SingleConnectionFactory.java:721)
at
org.apache.activemq.ActiveMQConnection$5.run(ActiveMQConnection.java:1967)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"DefaultMessageListenerContainer-502":
at
org.apache.activemq.ActiveMQConnection.getScheduler(ActiveMQConnection.java:2387)
- waiting to lock <0x0000000085048718> (a
org.apache.activemq.ActiveMQConnection)
at
org.apache.activemq.ActiveMQSession.getScheduler(ActiveMQSession.java:2250)
at
org.apache.activemq.ActiveMQMessageConsumer.rollback(ActiveMQMessageConsumer.java:1291)
- locked <0x000000008512c5d0> (a java.util.LinkedList)
- locked <0x000000008512c448> (a java.lang.Object)
at
org.apache.activemq.ActiveMQMessageConsumer$5.afterRollback(ActiveMQMessageConsumer.java:1094)
at
org.apache.activemq.TransactionContext.afterRollback(TransactionContext.java:162)
at
org.apache.activemq.TransactionContext.commit(TransactionContext.java:340)
at org.apache.activemq.ActiveMQSession.commit(ActiveMQSession.java:582)
at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.jms.connection.CachingConnectionFactory$CachedSessionInvocationHandler.invoke(CachingConnectionFactory.java:383)
at com.sun.proxy.$Proxy162.commit(Unknown Source)
at
org.springframework.jms.support.JmsUtils.commitIfNecessary(JmsUtils.java:218)
at
org.springframework.jms.listener.AbstractMessageListenerContainer.commitIfNecessary(AbstractMessageListenerContainer.java:776)
at
org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:680)
at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189)
at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179)
at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076)
at java.lang.Thread.run(Thread.java:748)
{noformat}
Thanks.
> Deadlock closing a connection due to an exception
> -------------------------------------------------
>
> Key: AMQ-6660
> URL: https://issues.apache.org/jira/browse/AMQ-6660
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.8.0
> Reporter: Dan Groves
> Assignee: Matt Pavlovich
> Priority: Major
> Labels: close-pending
> Fix For: AGING_TO_DIE
>
>
> While recovering from a period of network instability, our application
> deadlocked in ActiveMQ code.
> {noformat}
> Found one Java-level deadlock:
> =============================
> "ActiveMQ Connection Executor: tcp:///<ip address>:61616@51026":
> waiting to lock monitor 0x00007f60fc0060b8 (object 0x0000000773f00308, a
> java.lang.Object),
> which is held by "privateJmsInPriority.container-1"
> "privateJmsInPriority.container-1":
> waiting to lock monitor 0x00007f610840dbc8 (object 0x0000000773f99930, a
> org.apache.activemq.ActiveMQConnection),
> which is held by "ActiveMQ Connection Executor: tcp:///<ip
> address>:61616@51026"
> Java stack information for the threads listed above:
> ===================================================
> "ActiveMQ Connection Executor: tcp:///<ip address>:61616@51026":
> at
> org.apache.activemq.SimplePriorityMessageDispatchChannel.close(SimplePriorityMessageDispatchChannel.java:154)
> - waiting to lock <0x0000000773f00308> (a java.lang.Object)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispose(ActiveMQMessageConsumer.java:800)
> at
> org.apache.activemq.ActiveMQSession.dispose(ActiveMQSession.java:689)
> - locked <0x0000000773f003d8> (a org.apache.activemq.ActiveMQSession)
> at
> org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:659)
> - locked <0x0000000773f99930> (a
> org.apache.activemq.ActiveMQConnection)
> at
> org.springframework.jms.connection.SingleConnectionFactory.closeConnection(SingleConnectionFactory.java:456)
> at
> org.springframework.jms.connection.SingleConnectionFactory.resetConnection(SingleConnectionFactory.java:345)
> - locked <0x0000000773f9a590> (a java.lang.Object)
> at
> org.springframework.jms.connection.CachingConnectionFactory.resetConnection(CachingConnectionFactory.java:207)
> at
> org.springframework.jms.connection.SingleConnectionFactory.onException(SingleConnectionFactory.java:323)
> at
> org.springframework.jms.connection.SingleConnectionFactory$AggregatedExceptionListener.onException(SingleConnectionFactory.java:673)
> - locked <0x0000000773f9a590> (a java.lang.Object)
> at
> org.apache.activemq.ActiveMQConnection$5.run(ActiveMQConnection.java:1976)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "privateJmsInPriority.container-1":
> at
> org.apache.activemq.ActiveMQConnection.getScheduler(ActiveMQConnection.java:2519)
> - waiting to lock <0x0000000773f99930> (a
> org.apache.activemq.ActiveMQConnection)
> at
> org.apache.activemq.ActiveMQSession.getScheduler(ActiveMQSession.java:2074)
> at
> org.apache.activemq.ActiveMQMessageConsumer.rollback(ActiveMQMessageConsumer.java:1241)
> - locked <0x0000000773f01030> (a java.util.LinkedList)
> - locked <0x0000000773f00308> (a java.lang.Object)
> at
> org.apache.activemq.ActiveMQMessageConsumer$5.afterRollback(ActiveMQMessageConsumer.java:1032)
> at
> org.apache.activemq.TransactionContext.afterRollback(TransactionContext.java:157)
> at
> org.apache.activemq.TransactionContext.commit(TransactionContext.java:332)
> at
> org.apache.activemq.ActiveMQSession.commit(ActiveMQSession.java:561)
> at sun.reflect.GeneratedMethodAccessor897.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.jms.connection.CachingConnectionFactory$CachedSessionInvocationHandler.invoke(CachingConnectionFactory.java:386)
> at com.sun.proxy.$Proxy49.commit(Unknown Source)
> at
> org.springframework.jms.support.JmsUtils.commitIfNecessary(JmsUtils.java:217)
> at
> org.springframework.jms.listener.AbstractMessageListenerContainer.commitIfNecessary(AbstractMessageListenerContainer.java:761)
> at
> org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:665)
> at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:319)
> at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
> at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1166)
> at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1158)
> at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1055)
> at java.lang.Thread.run(Thread.java:745)
> Found 1 deadlock.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
