[
https://issues.apache.org/jira/browse/QPID-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Harvey updated QPID-4276:
--------------------------------
Description:
We have observed a deadlock on the Java client when the following are done
concurrently:
* An application thread closes a consumer
* The corresponding JMS connection is closed inside onMessage()
See attached stack trace for details.
We think this locking bug was introduced when fixing QPID-3911.
See thread dump excerpt below:
{noformat}
Found one Java-level deadlock:
=============================
"Thread-11-Consumer_0":
waiting to lock monitor 0x000000004342b4c0 (object 0x00000000ed1e6a40, a
java.lang.Object),
which is held by "Dispatcher-1-Conn-1"
"Dispatcher-1-Conn-1":
waiting to lock monitor 0x000000004342b568 (object 0x00000000ed060c48, a
java.lang.Object),
which is held by "Thread-11-Consumer_0"
Java stack information for the threads listed above:
===================================================
"Thread-11-Consumer_0":
at
org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:598)
- waiting to lock <0x00000000ed1e6a40> (a java.lang.Object)
- locked <0x00000000ed060c48> (a java.lang.Object)
at
org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:558)
at
org.apache.qpid.disttest.jms.ClientJmsDelegate.closeTestConsumer(ClientJmsDelegate.java:612)
at
org.apache.qpid.disttest.client.ConsumerParticipant.releaseResources(ConsumerParticipant.java:244)
at
org.apache.qpid.disttest.client.ParticipantExecutor$ParticipantRunnable.runParticipantAndSendResults(ParticipantExecutor.java:112)
at
org.apache.qpid.disttest.client.ParticipantExecutor$ParticipantRunnable.run(ParticipantExecutor.java:84)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
"Dispatcher-1-Conn-1":
at org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:857)
- waiting to lock <0x00000000ed060c48> (a java.lang.Object)
- locked <0x00000000ed060c58> (a java.lang.Object)
at org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:848)
- locked <0x00000000ed0bedd8> (a java.lang.Object)
- locked <0x00000000ed060c58> (a java.lang.Object)
at org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:848)
- locked <0x00000000ed1e6a40> (a java.lang.Object)
- locked <0x00000000ed060c58> (a java.lang.Object)
at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:832)
at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:823)
at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:818)
at
org.apache.qpid.disttest.jms.ClientJmsDelegate.tearDownTest(ClientJmsDelegate.java:574)
at org.apache.qpid.disttest.client.Client.tearDownTest(Client.java:179)
at
org.apache.qpid.disttest.client.ClientCommandVisitor.visit(ClientCommandVisitor.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.qpid.disttest.Visitor.invokeVisitMethod(Visitor.java:61)
at org.apache.qpid.disttest.Visitor.visit(Visitor.java:39)
at org.apache.qpid.disttest.message.Command.accept(Command.java:48)
at
org.apache.qpid.disttest.client.Client.processInstruction(Client.java:113)
at
org.apache.qpid.disttest.jms.ClientJmsDelegate$1.onMessage(ClientJmsDelegate.java:122)
at
org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:751)
at
org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:725)
at
org.apache.qpid.client.AMQSession$Dispatcher.notifyConsumer(AMQSession.java:3479)
at
org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3418)
- locked <0x00000000ed1a30f0> (a java.lang.Object)
- locked <0x00000000ed186400> (a java.lang.Object)
at
org.apache.qpid.client.AMQSession$Dispatcher.access$1000(AMQSession.java:3205)
at org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3198)
at
org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:54)
at
org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3341)
at java.lang.Thread.run(Thread.java:662)
Found 1 deadlock.
{noformat}
was:
We have observed a deadlock on the Java client when a consumer and its
connection are simultaneously closed.
See attached stack trace for details.
We think this locking bug was introduced when fixing QPID-3911.
See thread dump excerpt below:
{noformat}
Found one Java-level deadlock:
=============================
"Thread-11-Consumer_0":
waiting to lock monitor 0x000000004342b4c0 (object 0x00000000ed1e6a40, a
java.lang.Object),
which is held by "Dispatcher-1-Conn-1"
"Dispatcher-1-Conn-1":
waiting to lock monitor 0x000000004342b568 (object 0x00000000ed060c48, a
java.lang.Object),
which is held by "Thread-11-Consumer_0"
Java stack information for the threads listed above:
===================================================
"Thread-11-Consumer_0":
at
org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:598)
- waiting to lock <0x00000000ed1e6a40> (a java.lang.Object)
- locked <0x00000000ed060c48> (a java.lang.Object)
at
org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:558)
at
org.apache.qpid.disttest.jms.ClientJmsDelegate.closeTestConsumer(ClientJmsDelegate.java:612)
at
org.apache.qpid.disttest.client.ConsumerParticipant.releaseResources(ConsumerParticipant.java:244)
at
org.apache.qpid.disttest.client.ParticipantExecutor$ParticipantRunnable.runParticipantAndSendResults(ParticipantExecutor.java:112)
at
org.apache.qpid.disttest.client.ParticipantExecutor$ParticipantRunnable.run(ParticipantExecutor.java:84)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
"Dispatcher-1-Conn-1":
at org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:857)
- waiting to lock <0x00000000ed060c48> (a java.lang.Object)
- locked <0x00000000ed060c58> (a java.lang.Object)
at org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:848)
- locked <0x00000000ed0bedd8> (a java.lang.Object)
- locked <0x00000000ed060c58> (a java.lang.Object)
at org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:848)
- locked <0x00000000ed1e6a40> (a java.lang.Object)
- locked <0x00000000ed060c58> (a java.lang.Object)
at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:832)
at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:823)
at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:818)
at
org.apache.qpid.disttest.jms.ClientJmsDelegate.tearDownTest(ClientJmsDelegate.java:574)
at org.apache.qpid.disttest.client.Client.tearDownTest(Client.java:179)
at
org.apache.qpid.disttest.client.ClientCommandVisitor.visit(ClientCommandVisitor.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.qpid.disttest.Visitor.invokeVisitMethod(Visitor.java:61)
at org.apache.qpid.disttest.Visitor.visit(Visitor.java:39)
at org.apache.qpid.disttest.message.Command.accept(Command.java:48)
at
org.apache.qpid.disttest.client.Client.processInstruction(Client.java:113)
at
org.apache.qpid.disttest.jms.ClientJmsDelegate$1.onMessage(ClientJmsDelegate.java:122)
at
org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:751)
at
org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:725)
at
org.apache.qpid.client.AMQSession$Dispatcher.notifyConsumer(AMQSession.java:3479)
at
org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3418)
- locked <0x00000000ed1a30f0> (a java.lang.Object)
- locked <0x00000000ed186400> (a java.lang.Object)
at
org.apache.qpid.client.AMQSession$Dispatcher.access$1000(AMQSession.java:3205)
at org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3198)
at
org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:54)
at
org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3341)
at java.lang.Thread.run(Thread.java:662)
Found 1 deadlock.
{noformat}
> Java client - deadlock on concurrent close of consumer and connection
> ---------------------------------------------------------------------
>
> Key: QPID-4276
> URL: https://issues.apache.org/jira/browse/QPID-4276
> Project: Qpid
> Issue Type: Bug
> Components: Java Client
> Affects Versions: 0.16, 0.18
> Reporter: Philip Harvey
> Attachments: deadlock.stacktrace.txt
>
>
> We have observed a deadlock on the Java client when the following are done
> concurrently:
> * An application thread closes a consumer
> * The corresponding JMS connection is closed inside onMessage()
> See attached stack trace for details.
> We think this locking bug was introduced when fixing QPID-3911.
> See thread dump excerpt below:
> {noformat}
> Found one Java-level deadlock:
> =============================
> "Thread-11-Consumer_0":
> waiting to lock monitor 0x000000004342b4c0 (object 0x00000000ed1e6a40, a
> java.lang.Object),
> which is held by "Dispatcher-1-Conn-1"
> "Dispatcher-1-Conn-1":
> waiting to lock monitor 0x000000004342b568 (object 0x00000000ed060c48, a
> java.lang.Object),
> which is held by "Thread-11-Consumer_0"
> Java stack information for the threads listed above:
> ===================================================
> "Thread-11-Consumer_0":
> at
> org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:598)
> - waiting to lock <0x00000000ed1e6a40> (a java.lang.Object)
> - locked <0x00000000ed060c48> (a java.lang.Object)
> at
> org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:558)
> at
> org.apache.qpid.disttest.jms.ClientJmsDelegate.closeTestConsumer(ClientJmsDelegate.java:612)
> at
> org.apache.qpid.disttest.client.ConsumerParticipant.releaseResources(ConsumerParticipant.java:244)
> at
> org.apache.qpid.disttest.client.ParticipantExecutor$ParticipantRunnable.runParticipantAndSendResults(ParticipantExecutor.java:112)
> at
> org.apache.qpid.disttest.client.ParticipantExecutor$ParticipantRunnable.run(ParticipantExecutor.java:84)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> "Dispatcher-1-Conn-1":
> at
> org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:857)
> - waiting to lock <0x00000000ed060c48> (a java.lang.Object)
> - locked <0x00000000ed060c58> (a java.lang.Object)
> at
> org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:848)
> - locked <0x00000000ed0bedd8> (a java.lang.Object)
> - locked <0x00000000ed060c58> (a java.lang.Object)
> at
> org.apache.qpid.client.AMQConnection.doClose(AMQConnection.java:848)
> - locked <0x00000000ed1e6a40> (a java.lang.Object)
> - locked <0x00000000ed060c58> (a java.lang.Object)
> at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:832)
> at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:823)
> at org.apache.qpid.client.AMQConnection.close(AMQConnection.java:818)
> at
> org.apache.qpid.disttest.jms.ClientJmsDelegate.tearDownTest(ClientJmsDelegate.java:574)
> at
> org.apache.qpid.disttest.client.Client.tearDownTest(Client.java:179)
> at
> org.apache.qpid.disttest.client.ClientCommandVisitor.visit(ClientCommandVisitor.java:90)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.qpid.disttest.Visitor.invokeVisitMethod(Visitor.java:61)
> at org.apache.qpid.disttest.Visitor.visit(Visitor.java:39)
> at org.apache.qpid.disttest.message.Command.accept(Command.java:48)
> at
> org.apache.qpid.disttest.client.Client.processInstruction(Client.java:113)
> at
> org.apache.qpid.disttest.jms.ClientJmsDelegate$1.onMessage(ClientJmsDelegate.java:122)
> at
> org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:751)
> at
> org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:725)
> at
> org.apache.qpid.client.AMQSession$Dispatcher.notifyConsumer(AMQSession.java:3479)
> at
> org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3418)
> - locked <0x00000000ed1a30f0> (a java.lang.Object)
> - locked <0x00000000ed186400> (a java.lang.Object)
> at
> org.apache.qpid.client.AMQSession$Dispatcher.access$1000(AMQSession.java:3205)
> at org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3198)
> at
> org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:54)
> at
> org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3341)
> at java.lang.Thread.run(Thread.java:662)
> Found 1 deadlock.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
