Keith Wall created QPID-6813:
--------------------------------
Summary: Consumer tasks may run on the Broker thread leading to
deadlock
Key: QPID-6813
URL: https://issues.apache.org/jira/browse/QPID-6813
Project: Qpid
Issue Type: Bug
Components: Java Broker
Affects Versions: qpid-java-6.0
Reporter: Keith Wall
Assignee: Keith Wall
Fix For: qpid-java-6.0
Since QPID-6597, a consumer's tasks may run on the Broker's task executor
rather than the Virtualhost's as was intended. This may lead to deadlock.
The problem occurs because ACO discovers its task executor from its parents.
If a ACO has multiple parents, the chosen parent is non-deterministic. This
didn't used to matter because Consumers parents Queue and Session both shared a
Virtualhost ancestor, so the Virtualhost task executor was always selector.
However, QPID-6597, re-homed Connection (the parent of Session) to be a child
of Broker. Since that point the executor used has been non-deterministic.
Testing with very large numbers of connections exposed a deadlock between the
Broker and Virtualhost configuration thread.
{code:java}
protected AbstractConfiguredObject(final Map<Class<? extends ConfiguredObject>,
ConfiguredObject<?>> parents,
Map<String, Object> attributes)
{
this(parents, attributes,
parents.values().iterator().next().getChildExecutor());
}
{code}
{noformat}
"VirtualHostNode-default-Config" prio=10 tid=0x00007f48701a0000 nid=0xd714
waiting on condition [0x00007f48ac728000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e7ecd4a8> (a
com.google.common.util.concurrent.AbstractFuture$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
at
com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:285)
at
com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
at
org.apache.qpid.server.model.AbstractConfiguredObject.doSync(AbstractConfiguredObject.java:1937)
at
org.apache.qpid.server.model.AbstractConfiguredObject.open(AbstractConfiguredObject.java:452)
at
org.apache.qpid.server.queue.QueueConsumerImpl.<init>(QueueConsumerImpl.java:136)
at
org.apache.qpid.server.queue.AbstractQueue.addConsumerInternal(AbstractQueue.java:864)
at
org.apache.qpid.server.queue.AbstractQueue.access$200(AbstractQueue.java:119)
at
org.apache.qpid.server.queue.AbstractQueue$6.execute(AbstractQueue.java:695)
at
org.apache.qpid.server.queue.AbstractQueue$6.execute(AbstractQueue.java:691)
at
org.apache.qpid.server.configuration.updater.TaskExecutorImpl$TaskLoggingWrapper.execute(TaskExecutorImpl.java:270)
at
org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper$1.run(TaskExecutorImpl.java:342)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at
org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper.call(TaskExecutorImpl.java:335)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- <0x00000000e5953fb0> (a
java.util.concurrent.ThreadPoolExecutor$Worker)
"Broker-Config" prio=10 tid=0x00007f49c8f67000 nid=0xd70d waiting on condition
[0x00007f48ac95e000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000ea9a2940> (a
com.google.common.util.concurrent.AbstractFuture$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
at
com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:285)
at
com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
at
org.apache.qpid.server.model.AbstractConfiguredObject.doSync(AbstractConfiguredObject.java:1937)
at
org.apache.qpid.server.virtualhost.AbstractVirtualHost.removeQueue(AbstractVirtualHost.java:734)
at
org.apache.qpid.server.queue.AbstractQueue$7.run(AbstractQueue.java:971)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at
org.apache.qpid.server.queue.AbstractQueue.unregisterConsumer(AbstractQueue.java:966)
at
org.apache.qpid.server.queue.QueueConsumerImpl.onClose(QueueConsumerImpl.java:282)
at
org.apache.qpid.server.model.AbstractConfiguredObject$5$1.run(AbstractConfiguredObject.java:666)
at
org.apache.qpid.server.model.AbstractConfiguredObject$19.onSuccess(AbstractConfiguredObject.java:2145)
at com.google.common.util.concurrent.Futures$6.run(Futures.java:1319)
at
org.apache.qpid.server.configuration.updater.TaskExecutorImpl$ImmediateIfSameThreadExecutor.execute(TaskExecutorImpl.java:422)
at
org.apache.qpid.server.configuration.updater.TaskExecutorImpl.execute(TaskExecutorImpl.java:172)
at
com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
at
com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:101)
at
com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:170)
at
com.google.common.util.concurrent.Futures.addCallback(Futures.java:1322)
at
org.apache.qpid.server.model.AbstractConfiguredObject.doAfter(AbstractConfiguredObject.java:2138)
at
org.apache.qpid.server.model.AbstractConfiguredObject$ChainedSettableFuture.then(AbstractConfiguredObject.java:2200)
at
org.apache.qpid.server.model.AbstractConfiguredObject$5.execute(AbstractConfiguredObject.java:654)
at
org.apache.qpid.server.model.AbstractConfiguredObject$5.execute(AbstractConfiguredObject.java:645)
at
org.apache.qpid.server.model.AbstractConfiguredObject$2.execute(AbstractConfiguredObject.java:520)
at
org.apache.qpid.server.model.AbstractConfiguredObject$2.execute(AbstractConfiguredObject.java:513)
at
org.apache.qpid.server.configuration.updater.TaskExecutorImpl$TaskLoggingWrapper.execute(TaskExecutorImpl.java:270)
at
org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper$1.run(TaskExecutorImpl.java:342)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at
org.apache.qpid.server.configuration.updater.TaskExecutorImpl$CallableWrapper.call(TaskExecutorImpl.java:335)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- <0x00000000e59603c8> (a
java.util.concurrent.ThreadPoolExecutor$Worker)
- <0x00000000ea413e18> (a
java.util.concurrent.locks.ReentrantLock$NonfairSync)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
