Keith Wall created QPID-4044:
--------------------------------
Summary: Python tests expose a deadlock in distributed
transactions support within Java Broker
Key: QPID-4044
URL: https://issues.apache.org/jira/browse/QPID-4044
Project: Qpid
Issue Type: Bug
Components: Java Broker
Affects Versions: 0.17
Reporter: Keith Wall
The Python-Java-Test job on Apache Jenkins CI has produced a deadlocked Java
Broker on at least two occasions over the last month. Examining the latest
occurrence, we can see that the Java Broker has deadlocked in the distributed
transaction code during shutdown.
The Jstack is as follows:
{code}
Full thread dump Java HotSpot(TM) Server VM (20.2-b06 mixed mode):
"Attach Listener" daemon prio=10 tid=0x08899c00 nid=0x561b waiting on condition
[0x00000000]
java.lang.Thread.State: RUNNABLE
"SIGTERM handler" daemon prio=10 tid=0x08893800 nid=0x65d6 waiting for monitor
entry [0xabafe000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.Shutdown.exit(Shutdown.java:168)
- waiting to lock <0xef6e36c8> (a java.lang.Class for
java.lang.Shutdown)
at java.lang.Terminator$1.handle(Terminator.java:35)
at sun.misc.Signal$1.run(Signal.java:195)
at java.lang.Thread.run(Thread.java:662)
"QpidBrokerShutdownHook" prio=10 tid=0x08894000 nid=0x65c5 in Object.wait()
[0xa9870000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb7448e40> (a java.lang.Object)
at org.apache.qpid.transport.util.Waiter.await(Waiter.java:54)
at org.apache.qpid.transport.Connection.close(Connection.java:622)
- locked <0xb7448e40> (a java.lang.Object)
at
org.apache.qpid.server.transport.ServerConnection.close(ServerConnection.java:286)
at
org.apache.qpid.server.connection.ConnectionRegistry.closeConnection(ConnectionRegistry.java:70)
at
org.apache.qpid.server.connection.ConnectionRegistry.close(ConnectionRegistry.java:62)
at
org.apache.qpid.server.connection.ConnectionRegistry.close(ConnectionRegistry.java:50)
at
org.apache.qpid.server.virtualhost.VirtualHostImpl.close(VirtualHostImpl.java:521)
at
org.apache.qpid.server.virtualhost.VirtualHostRegistry.close(VirtualHostRegistry.java:105)
at
org.apache.qpid.server.registry.ApplicationRegistry.close(ApplicationRegistry.java:460)
at
org.apache.qpid.server.registry.ApplicationRegistry.close(ApplicationRegistry.java:490)
at
org.apache.qpid.server.registry.ApplicationRegistry.remove(ApplicationRegistry.java:248)
at org.apache.qpid.server.Broker.shutdown(Broker.java:83)
at org.apache.qpid.server.Broker$ShutdownService.run(Broker.java:553)
at java.lang.Thread.run(Thread.java:662)
"SIGTERM handler" daemon prio=10 tid=0x089ef800 nid=0x65c2 in Object.wait()
[0xa981f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb7452ea0> (a java.lang.Thread)
at java.lang.Thread.join(Thread.java:1186)
- locked <0xb7452ea0> (a java.lang.Thread)
at java.lang.Thread.join(Thread.java:1239)
at
java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
at
java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
at java.lang.Shutdown.runHooks(Shutdown.java:79)
at java.lang.Shutdown.sequence(Shutdown.java:123)
at java.lang.Shutdown.exit(Shutdown.java:168)
- locked <0xef6e36c8> (a java.lang.Class for java.lang.Shutdown)
at java.lang.Terminator$1.handle(Terminator.java:35)
at sun.misc.Signal$1.run(Signal.java:195)
at java.lang.Thread.run(Thread.java:662)
"IoReceiver - /0:0:0:0:0:0:0:1:52037" daemon prio=10 tid=0xaa9fbc00 nid=0x3d1a
waiting for monitor entry [0xa98c1000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.qpid.server.txn.DtxRegistry.getBranch(DtxRegistry.java:100)
- waiting to lock <0xb7476d58> (a
org.apache.qpid.server.txn.DtxRegistry)
at
org.apache.qpid.server.txn.DistributedTransaction.start(DistributedTransaction.java:163)
at
org.apache.qpid.server.transport.ServerSession.startDtx(ServerSession.java:495)
at
org.apache.qpid.server.transport.ServerSessionDelegate.dtxStart(ServerSessionDelegate.java:459)
at
org.apache.qpid.server.transport.ServerSessionDelegate.dtxStart(ServerSessionDelegate.java:74)
at org.apache.qpid.transport.DtxStart.dispatch(DtxStart.java:89)
at
org.apache.qpid.transport.SessionDelegate.command(SessionDelegate.java:55)
at
org.apache.qpid.server.transport.ServerSessionDelegate.command(ServerSessionDelegate.java:98)
at
org.apache.qpid.server.transport.ServerSessionDelegate.command(ServerSessionDelegate.java:74)
at org.apache.qpid.transport.Method.delegate(Method.java:159)
at org.apache.qpid.transport.Session.received(Session.java:584)
at org.apache.qpid.transport.Connection.dispatch(Connection.java:421)
at
org.apache.qpid.transport.ConnectionDelegate.handle(ConnectionDelegate.java:64)
at
org.apache.qpid.transport.ConnectionDelegate.handle(ConnectionDelegate.java:40)
at
org.apache.qpid.transport.MethodDelegate.dtxStart(MethodDelegate.java:161)
at org.apache.qpid.transport.DtxStart.dispatch(DtxStart.java:89)
at
org.apache.qpid.transport.ConnectionDelegate.command(ConnectionDelegate.java:54)
at
org.apache.qpid.transport.ConnectionDelegate.command(ConnectionDelegate.java:40)
at org.apache.qpid.transport.Method.delegate(Method.java:159)
at org.apache.qpid.transport.Connection.received(Connection.java:376)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:225)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:59)
at org.apache.qpid.transport.network.Assembler.emit(Assembler.java:97)
at
org.apache.qpid.transport.network.Assembler.assemble(Assembler.java:198)
at org.apache.qpid.transport.network.Assembler.frame(Assembler.java:131)
at org.apache.qpid.transport.network.Frame.delegate(Frame.java:128)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:102)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:44)
at
org.apache.qpid.transport.network.InputHandler.next(InputHandler.java:189)
at
org.apache.qpid.transport.network.InputHandler.received(InputHandler.java:105)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:109)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:42)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:118)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:37)
at
org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:152)
at java.lang.Thread.run(Thread.java:662)
"IoSender - /0:0:0:0:0:0:0:1:52037" daemon prio=10 tid=0xaa9f9c00 nid=0x3d19 in
Object.wait() [0xa9912000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb73f8f18> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at org.apache.qpid.transport.network.io.IoSender.run(IoSender.java:284)
- locked <0xb73f8f18> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:662)
"IoReceiver - /0:0:0:0:0:0:0:1:49533" daemon prio=10 tid=0xac424400 nid=0x3a7f
waiting for monitor entry [0xa9c8d000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.qpid.server.txn.DtxBranch.isAssociated(DtxBranch.java:141)
- waiting to lock <0xb7592628> (a org.apache.qpid.server.txn.DtxBranch)
at
org.apache.qpid.server.txn.DtxRegistry.endAssociations(DtxRegistry.java:315)
- locked <0xb7476d58> (a org.apache.qpid.server.txn.DtxRegistry)
at
org.apache.qpid.server.transport.ServerSession.onClose(ServerSession.java:380)
at
org.apache.qpid.server.transport.ServerSessionDelegate.closed(ServerSessionDelegate.java:1573)
at org.apache.qpid.transport.Session.closed(Session.java:1096)
- locked <0xb7410418> (a java.lang.Object)
at
org.apache.qpid.transport.ConnectionDelegate.sessionDetached(ConnectionDelegate.java:105)
at
org.apache.qpid.transport.ConnectionDelegate.sessionDetached(ConnectionDelegate.java:40)
at
org.apache.qpid.transport.SessionDetached.dispatch(SessionDetached.java:91)
at
org.apache.qpid.transport.ConnectionDelegate.control(ConnectionDelegate.java:49)
at
org.apache.qpid.transport.ConnectionDelegate.control(ConnectionDelegate.java:40)
at org.apache.qpid.transport.Method.delegate(Method.java:163)
at org.apache.qpid.transport.Connection.received(Connection.java:376)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:225)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:59)
at org.apache.qpid.transport.network.Assembler.emit(Assembler.java:97)
at
org.apache.qpid.transport.network.Assembler.assemble(Assembler.java:183)
at org.apache.qpid.transport.network.Assembler.frame(Assembler.java:131)
at org.apache.qpid.transport.network.Frame.delegate(Frame.java:128)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:102)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:44)
at
org.apache.qpid.transport.network.InputHandler.next(InputHandler.java:189)
at
org.apache.qpid.transport.network.InputHandler.received(InputHandler.java:105)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:109)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:42)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:118)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:37)
at
org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:152)
at java.lang.Thread.run(Thread.java:662)
"IoSender - /0:0:0:0:0:0:0:1:49533" daemon prio=10 tid=0xac423800 nid=0x3a7e in
Object.wait() [0xa9cde000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb7406af0> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at org.apache.qpid.transport.network.io.IoSender.run(IoSender.java:284)
- locked <0xb7406af0> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:662)
"IoReceiver - /0:0:0:0:0:0:0:1:49532" daemon prio=10 tid=0xab514400 nid=0x3a7c
waiting for monitor entry [0xabaad000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.qpid.server.txn.DtxRegistry.unregisterBranch(DtxRegistry.java:116)
- waiting to lock <0xb7476d58> (a
org.apache.qpid.server.txn.DtxRegistry)
at org.apache.qpid.server.txn.DtxRegistry.rollback(DtxRegistry.java:228)
- locked <0xb7592628> (a org.apache.qpid.server.txn.DtxBranch)
at
org.apache.qpid.server.transport.ServerSession.rollbackDtx(ServerSession.java:543)
at
org.apache.qpid.server.transport.ServerSessionDelegate.dtxRollback(ServerSessionDelegate.java:640)
at
org.apache.qpid.server.transport.ServerSessionDelegate.dtxRollback(ServerSessionDelegate.java:74)
at org.apache.qpid.transport.DtxRollback.dispatch(DtxRollback.java:87)
at
org.apache.qpid.transport.SessionDelegate.command(SessionDelegate.java:55)
at
org.apache.qpid.server.transport.ServerSessionDelegate.command(ServerSessionDelegate.java:98)
at
org.apache.qpid.server.transport.ServerSessionDelegate.command(ServerSessionDelegate.java:74)
at org.apache.qpid.transport.Method.delegate(Method.java:159)
at org.apache.qpid.transport.Session.received(Session.java:584)
at org.apache.qpid.transport.Connection.dispatch(Connection.java:421)
at
org.apache.qpid.transport.ConnectionDelegate.handle(ConnectionDelegate.java:64)
at
org.apache.qpid.transport.ConnectionDelegate.handle(ConnectionDelegate.java:40)
at
org.apache.qpid.transport.MethodDelegate.dtxRollback(MethodDelegate.java:182)
at org.apache.qpid.transport.DtxRollback.dispatch(DtxRollback.java:87)
at
org.apache.qpid.transport.ConnectionDelegate.command(ConnectionDelegate.java:54)
at
org.apache.qpid.transport.ConnectionDelegate.command(ConnectionDelegate.java:40)
at org.apache.qpid.transport.Method.delegate(Method.java:159)
at org.apache.qpid.transport.Connection.received(Connection.java:376)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:225)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:59)
at org.apache.qpid.transport.network.Assembler.emit(Assembler.java:97)
at
org.apache.qpid.transport.network.Assembler.assemble(Assembler.java:198)
at org.apache.qpid.transport.network.Assembler.frame(Assembler.java:131)
at org.apache.qpid.transport.network.Frame.delegate(Frame.java:128)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:102)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:44)
at
org.apache.qpid.transport.network.InputHandler.next(InputHandler.java:189)
at
org.apache.qpid.transport.network.InputHandler.received(InputHandler.java:105)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:109)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:42)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:118)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:37)
at
org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:152)
at java.lang.Thread.run(Thread.java:662)
"test:UpdateTask" prio=10 tid=0x0880dc00 nid=0x3a71 waiting on condition
[0xa9d80000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edad8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-32" prio=10 tid=0x08a43000 nid=0x39e1 waiting on condition
[0xa9e22000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-31" prio=10 tid=0x08a4cc00 nid=0x39e0 waiting on condition
[0xa9e73000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-30" prio=10 tid=0x08a4b400 nid=0x39df waiting on condition
[0xa9ec4000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-29" prio=10 tid=0x08a4a000 nid=0x39de waiting on condition
[0xa9f15000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-28" prio=10 tid=0x08a3e800 nid=0x39da waiting on condition
[0xa9f66000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-27" prio=10 tid=0x088c6400 nid=0x39d9 waiting on condition
[0xa9fb7000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-26" prio=10 tid=0x088c5000 nid=0x39d8 waiting on condition
[0xaa008000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-25" prio=10 tid=0x088c4000 nid=0x39d7 waiting on condition
[0xaa059000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-24" prio=10 tid=0x088c2800 nid=0x39d6 waiting on condition
[0xaa0aa000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-23" prio=10 tid=0x088c0c00 nid=0x39d5 waiting on condition
[0xaa0fb000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-22" prio=10 tid=0x088bf800 nid=0x39d4 waiting on condition
[0xaa14c000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-21" prio=10 tid=0x088be000 nid=0x39d0 waiting on condition
[0xaa19d000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-20" prio=10 tid=0x088bc800 nid=0x39cf waiting on condition
[0xaa1ee000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-19" prio=10 tid=0x088a2400 nid=0x39ce waiting on condition
[0xaa23f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-18" prio=10 tid=0x088a0c00 nid=0x39cd waiting on condition
[0xaa290000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-17" prio=10 tid=0x0889f400 nid=0x39cc waiting on condition
[0xaa2e1000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-16" prio=10 tid=0x0889dc00 nid=0x39cb waiting on condition
[0xaa332000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-15" prio=10 tid=0x0889c800 nid=0x39ca waiting on condition
[0xaa383000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-14" prio=10 tid=0x088f9400 nid=0x39c9 waiting on condition
[0xaa3d4000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-13" prio=10 tid=0x088f8400 nid=0x39c8 waiting on condition
[0xaa425000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-12" prio=10 tid=0x088f7400 nid=0x39c7 waiting on condition
[0xaa476000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-11" prio=10 tid=0x088f5400 nid=0x39b5 waiting on condition
[0xaa4c7000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-10" prio=10 tid=0x088f4400 nid=0x39b4 waiting on condition
[0xaa518000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-9" prio=10 tid=0x088d1800 nid=0x39aa waiting on condition
[0xaa569000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-8" prio=10 tid=0x088d0400 nid=0x39a9 waiting on condition
[0xaa5ba000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-7" prio=10 tid=0x088cf800 nid=0x39a8 waiting on condition
[0xaa60b000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-6" prio=10 tid=0x08a36c00 nid=0x39a7 waiting on condition
[0xaa65c000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-5" prio=10 tid=0x08a35800 nid=0x39a6 waiting on condition
[0xaa6ad000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-4" prio=10 tid=0x08902c00 nid=0x39a3 waiting on condition
[0xaa6fe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-3" prio=10 tid=0x08a3c800 nid=0x39a2 waiting on condition
[0xab75c000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-2" prio=10 tid=0x089fd800 nid=0x39a1 waiting on condition
[0xab7ad000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"pool-3-thread-1" prio=10 tid=0x08a39000 nid=0x39a0 waiting on condition
[0xab7fe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edac0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"IoSender - /127.0.0.1:41936" daemon prio=10 tid=0x089f9c00 nid=0x3963 in
Object.wait() [0xab9e9000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb74745c8> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at org.apache.qpid.transport.network.io.IoSender.run(IoSender.java:284)
- locked <0xb74745c8> (a java.lang.Object)
at java.lang.Thread.run(Thread.java:662)
"DestroyJavaVM" prio=10 tid=0xabbf2400 nid=0x3917 waiting on condition
[0x00000000]
java.lang.Thread.State: RUNNABLE
"test:UpdateTask" prio=10 tid=0xac0f0000 nid=0x394e waiting on condition
[0xabcd4000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edad8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"test:UpdateTask" prio=10 tid=0xac0df800 nid=0x394d waiting on condition
[0xabd25000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edad8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"development:UpdateTask" prio=10 tid=0xac0fd000 nid=0x394c waiting on condition
[0xabd76000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edb08> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"development:VirtualHostHouseKeepingTask" prio=10 tid=0xac0f9c00 nid=0x394b
waiting on condition [0xabdc7000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xb73edb08> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:662)
"GC Daemon" daemon prio=10 tid=0xac142c00 nid=0x394a in Object.wait()
[0xabe18000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb733c0d8> (a sun.misc.GC$LatencyLock)
at sun.misc.GC$Daemon.run(GC.java:100)
- locked <0xb733c0d8> (a sun.misc.GC$LatencyLock)
"RMI Reaper" prio=10 tid=0x088b8800 nid=0x3949 in Object.wait() [0xabe69000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb7337100> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0xb7337100> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:333)
at java.lang.Thread.run(Thread.java:662)
"RMI TCP Accept-25100" daemon prio=10 tid=0x088b7800 nid=0x3948 runnable
[0xabeba000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
- locked <0xb733bcf8> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at java.net.ServerSocket.accept(ServerSocket.java:430)
at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
at java.lang.Thread.run(Thread.java:662)
"RMI TCP Accept-25000" daemon prio=10 tid=0x08889400 nid=0x3947 runnable
[0xabf0b000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
- locked <0xb7337b10> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:462)
at
org.apache.qpid.server.management.JMXManagedObjectRegistry$CustomRMIServerSocketFactory$NoLocalAddressServerSocket.accept(JMXManagedObjectRegistry.java:391)
at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
at java.lang.Thread.run(Thread.java:662)
"FelixPackageAdmin" daemon prio=10 tid=0xac0ec000 nid=0x392e in Object.wait()
[0xabf5c000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb733c650> (a
org.apache.felix.framework.PackageAdminImpl)
at java.lang.Object.wait(Object.java:485)
at
org.apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:316)
- locked <0xb733c650> (a org.apache.felix.framework.PackageAdminImpl)
at java.lang.Thread.run(Thread.java:662)
"FelixStartLevel" daemon prio=10 tid=0xac0ea800 nid=0x392d in Object.wait()
[0xabfad000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb733e1e0> (a java.util.ArrayList)
at java.lang.Object.wait(Object.java:485)
at
org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:242)
- locked <0xb733e1e0> (a java.util.ArrayList)
at java.lang.Thread.run(Thread.java:662)
"FelixDispatchQueue" prio=10 tid=0xac0b2c00 nid=0x392c in Object.wait()
[0xabffe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb733d280> (a java.util.ArrayList)
at java.lang.Object.wait(Object.java:485)
at
org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:917)
- locked <0xb733d280> (a java.util.ArrayList)
at
org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:54)
at
org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:106)
at java.lang.Thread.run(Thread.java:662)
"Low Memory Detector" daemon prio=10 tid=0xf3602400 nid=0x392a runnable
[0x00000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" daemon prio=10 tid=0xf3600800 nid=0x3929 waiting on
condition [0x00000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" daemon prio=10 tid=0x08740c00 nid=0x3928 waiting on
condition [0x00000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x0873f400 nid=0x3927 runnable
[0x00000000]
java.lang.Thread.State: RUNNABLE
"Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x0873dc00
nid=0x3926 waiting on condition [0x00000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0x08730000 nid=0x3925 in Object.wait()
[0xac87d000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb733f0b0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0xb733f0b0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0x0872b400 nid=0x3924 in Object.wait()
[0xac8ce000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xb733c6d8> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0xb733c6d8> (a java.lang.ref.Reference$Lock)
"VM Thread" prio=10 tid=0x08727800 nid=0x3923 runnable
"Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x0850ac00 nid=0x3918
runnable
"Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x0850c400 nid=0x3919
runnable
"Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x0850dc00 nid=0x391a
runnable
"Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x0850f000 nid=0x391b
runnable
"Gang worker#4 (Parallel GC Threads)" prio=10 tid=0x08510800 nid=0x391c
runnable
"Gang worker#5 (Parallel GC Threads)" prio=10 tid=0x08511c00 nid=0x391d
runnable
"Gang worker#6 (Parallel GC Threads)" prio=10 tid=0x08513400 nid=0x391e
runnable
"Gang worker#7 (Parallel GC Threads)" prio=10 tid=0x08514800 nid=0x391f
runnable
"Concurrent Mark-Sweep GC Thread" prio=10 tid=0x0869cc00 nid=0x3922 runnable
"Gang worker#0 (Parallel CMS Threads)" prio=10 tid=0x08619c00 nid=0x3920
runnable
"Gang worker#1 (Parallel CMS Threads)" prio=10 tid=0x0861b000 nid=0x3921
runnable
"VM Periodic Task Thread" prio=10 tid=0xf360cc00 nid=0x392b waiting on
condition
JNI global references: 1922
Found one Java-level deadlock:
=============================
"IoReceiver - /0:0:0:0:0:0:0:1:52037":
waiting to lock monitor 0x08a1e150 (object 0xb7476d58, a
org.apache.qpid.server.txn.DtxRegistry),
which is held by "IoReceiver - /0:0:0:0:0:0:0:1:49533"
"IoReceiver - /0:0:0:0:0:0:0:1:49533":
waiting to lock monitor 0x08a1e088 (object 0xb7592628, a
org.apache.qpid.server.txn.DtxBranch),
which is held by "IoReceiver - /0:0:0:0:0:0:0:1:49532"
"IoReceiver - /0:0:0:0:0:0:0:1:49532":
waiting to lock monitor 0x08a1e150 (object 0xb7476d58, a
org.apache.qpid.server.txn.DtxRegistry),
which is held by "IoReceiver - /0:0:0:0:0:0:0:1:49533"
Java stack information for the threads listed above:
===================================================
"IoReceiver - /0:0:0:0:0:0:0:1:52037":
at
org.apache.qpid.server.txn.DtxRegistry.getBranch(DtxRegistry.java:100)
- waiting to lock <0xb7476d58> (a
org.apache.qpid.server.txn.DtxRegistry)
at
org.apache.qpid.server.txn.DistributedTransaction.start(DistributedTransaction.java:163)
at
org.apache.qpid.server.transport.ServerSession.startDtx(ServerSession.java:495)
at
org.apache.qpid.server.transport.ServerSessionDelegate.dtxStart(ServerSessionDelegate.java:459)
at
org.apache.qpid.server.transport.ServerSessionDelegate.dtxStart(ServerSessionDelegate.java:74)
at org.apache.qpid.transport.DtxStart.dispatch(DtxStart.java:89)
at
org.apache.qpid.transport.SessionDelegate.command(SessionDelegate.java:55)
at
org.apache.qpid.server.transport.ServerSessionDelegate.command(ServerSessionDelegate.java:98)
at
org.apache.qpid.server.transport.ServerSessionDelegate.command(ServerSessionDelegate.java:74)
at org.apache.qpid.transport.Method.delegate(Method.java:159)
at org.apache.qpid.transport.Session.received(Session.java:584)
at org.apache.qpid.transport.Connection.dispatch(Connection.java:421)
at
org.apache.qpid.transport.ConnectionDelegate.handle(ConnectionDelegate.java:64)
at
org.apache.qpid.transport.ConnectionDelegate.handle(ConnectionDelegate.java:40)
at
org.apache.qpid.transport.MethodDelegate.dtxStart(MethodDelegate.java:161)
at org.apache.qpid.transport.DtxStart.dispatch(DtxStart.java:89)
at
org.apache.qpid.transport.ConnectionDelegate.command(ConnectionDelegate.java:54)
at
org.apache.qpid.transport.ConnectionDelegate.command(ConnectionDelegate.java:40)
at org.apache.qpid.transport.Method.delegate(Method.java:159)
at org.apache.qpid.transport.Connection.received(Connection.java:376)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:225)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:59)
at org.apache.qpid.transport.network.Assembler.emit(Assembler.java:97)
at
org.apache.qpid.transport.network.Assembler.assemble(Assembler.java:198)
at org.apache.qpid.transport.network.Assembler.frame(Assembler.java:131)
at org.apache.qpid.transport.network.Frame.delegate(Frame.java:128)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:102)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:44)
at
org.apache.qpid.transport.network.InputHandler.next(InputHandler.java:189)
at
org.apache.qpid.transport.network.InputHandler.received(InputHandler.java:105)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:109)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:42)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:118)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:37)
at
org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:152)
at java.lang.Thread.run(Thread.java:662)
"IoReceiver - /0:0:0:0:0:0:0:1:49533":
at org.apache.qpid.server.txn.DtxBranch.isAssociated(DtxBranch.java:141)
- waiting to lock <0xb7592628> (a org.apache.qpid.server.txn.DtxBranch)
at
org.apache.qpid.server.txn.DtxRegistry.endAssociations(DtxRegistry.java:315)
- locked <0xb7476d58> (a org.apache.qpid.server.txn.DtxRegistry)
at
org.apache.qpid.server.transport.ServerSession.onClose(ServerSession.java:380)
at
org.apache.qpid.server.transport.ServerSessionDelegate.closed(ServerSessionDelegate.java:1573)
at org.apache.qpid.transport.Session.closed(Session.java:1096)
- locked <0xb7410418> (a java.lang.Object)
at
org.apache.qpid.transport.ConnectionDelegate.sessionDetached(ConnectionDelegate.java:105)
at
org.apache.qpid.transport.ConnectionDelegate.sessionDetached(ConnectionDelegate.java:40)
at
org.apache.qpid.transport.SessionDetached.dispatch(SessionDetached.java:91)
at
org.apache.qpid.transport.ConnectionDelegate.control(ConnectionDelegate.java:49)
at
org.apache.qpid.transport.ConnectionDelegate.control(ConnectionDelegate.java:40)
at org.apache.qpid.transport.Method.delegate(Method.java:163)
at org.apache.qpid.transport.Connection.received(Connection.java:376)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:225)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:59)
at org.apache.qpid.transport.network.Assembler.emit(Assembler.java:97)
at
org.apache.qpid.transport.network.Assembler.assemble(Assembler.java:183)
at org.apache.qpid.transport.network.Assembler.frame(Assembler.java:131)
at org.apache.qpid.transport.network.Frame.delegate(Frame.java:128)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:102)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:44)
at
org.apache.qpid.transport.network.InputHandler.next(InputHandler.java:189)
at
org.apache.qpid.transport.network.InputHandler.received(InputHandler.java:105)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:109)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:42)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:118)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:37)
at
org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:152)
at java.lang.Thread.run(Thread.java:662)
"IoReceiver - /0:0:0:0:0:0:0:1:49532":
at
org.apache.qpid.server.txn.DtxRegistry.unregisterBranch(DtxRegistry.java:116)
- waiting to lock <0xb7476d58> (a
org.apache.qpid.server.txn.DtxRegistry)
at org.apache.qpid.server.txn.DtxRegistry.rollback(DtxRegistry.java:228)
- locked <0xb7592628> (a org.apache.qpid.server.txn.DtxBranch)
at
org.apache.qpid.server.transport.ServerSession.rollbackDtx(ServerSession.java:543)
at
org.apache.qpid.server.transport.ServerSessionDelegate.dtxRollback(ServerSessionDelegate.java:640)
at
org.apache.qpid.server.transport.ServerSessionDelegate.dtxRollback(ServerSessionDelegate.java:74)
at org.apache.qpid.transport.DtxRollback.dispatch(DtxRollback.java:87)
at
org.apache.qpid.transport.SessionDelegate.command(SessionDelegate.java:55)
at
org.apache.qpid.server.transport.ServerSessionDelegate.command(ServerSessionDelegate.java:98)
at
org.apache.qpid.server.transport.ServerSessionDelegate.command(ServerSessionDelegate.java:74)
at org.apache.qpid.transport.Method.delegate(Method.java:159)
at org.apache.qpid.transport.Session.received(Session.java:584)
at org.apache.qpid.transport.Connection.dispatch(Connection.java:421)
at
org.apache.qpid.transport.ConnectionDelegate.handle(ConnectionDelegate.java:64)
at
org.apache.qpid.transport.ConnectionDelegate.handle(ConnectionDelegate.java:40)
at
org.apache.qpid.transport.MethodDelegate.dtxRollback(MethodDelegate.java:182)
at org.apache.qpid.transport.DtxRollback.dispatch(DtxRollback.java:87)
at
org.apache.qpid.transport.ConnectionDelegate.command(ConnectionDelegate.java:54)
at
org.apache.qpid.transport.ConnectionDelegate.command(ConnectionDelegate.java:40)
at org.apache.qpid.transport.Method.delegate(Method.java:159)
at org.apache.qpid.transport.Connection.received(Connection.java:376)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:225)
at
org.apache.qpid.server.transport.ServerConnection.received(ServerConnection.java:59)
at org.apache.qpid.transport.network.Assembler.emit(Assembler.java:97)
at
org.apache.qpid.transport.network.Assembler.assemble(Assembler.java:198)
at org.apache.qpid.transport.network.Assembler.frame(Assembler.java:131)
at org.apache.qpid.transport.network.Frame.delegate(Frame.java:128)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:102)
at
org.apache.qpid.transport.network.Assembler.received(Assembler.java:44)
at
org.apache.qpid.transport.network.InputHandler.next(InputHandler.java:189)
at
org.apache.qpid.transport.network.InputHandler.received(InputHandler.java:105)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:109)
at
org.apache.qpid.server.protocol.ProtocolEngine_0_10.received(ProtocolEngine_0_10.java:42)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:118)
at
org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:37)
at
org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:152)
at java.lang.Thread.run(Thread.java:662)
Found 1 deadlock.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
