Michael Yara created AMQ-4935:
---------------------------------
Summary: Deadlock caused by Re-entrant lock with no locking thread
Key: AMQ-4935
URL: https://issues.apache.org/jira/browse/AMQ-4935
Project: ActiveMQ
Issue Type: Bug
Components: activemq-pool
Affects Versions: 5.9.0
Environment: Oracle JRE 1.7.0 update 45
Reporter: Michael Yara
We have a distributed application that uses ActiveMQ for communication between
nodes. We are currently using ActiveMQ 5.9.0 running with Oracle JRE 1.7.0
update 45.
We are seeing some strange deadlock behavior in the ActiveMQ ThreadPoolExecutor
when trying to serialize instances of ConcurrentHashMap. The thread dump can be
seen at the following link:
http://pastebin.com/wmNxSF4p
We can see that there are two threads that are blocked waiting for a lock on
some of the segments in a ConcurrentHashMap yet the thread dump does not
indicate any threads that currently hold a lock on those objects. This lock
does not occur all the time and can take around 5-10 messages before the
deadlock occurs. I have been able to recreate this issue on two different
physical machines.
Our test environment that generated this thread dump is as follows. Node A is
the instance that thread dump was taken from. For testing both nodes are on the
same physical machine.
Node A:
* In its own JVM instance.
* Runs an embedded ActiveMQ broker.
* Performs communication via vm transport and sends ObjectMessages via a
TopicConnection.
* Has copyMessageOnSend=false
* Has ObjectMessageSerializationDefered=true
Node B:
* In its own JVM instance.
* Connects to Node A's broker via tcp transport.
* Sends BytesMessages serialized with Kryo and sent via a TopicConnection.
* Has copyMessageOnSend=false
* Has ObjectMessageSerializationDefered=false
Both nodes are set up to detect the type of incoming messages and deserialize
them appropriately with kryo if it receives a BytesMessage or default
serialization if it receives an ObjectMessage (handled by ActiveMQ).
The only thing I have found resembling this issue is the following link. I am
no sure if it is related to this exact problem or not.
http://appcrawler.com/wordpress/2013/05/06/one-way-to-tell-if-a-thread-pool-is-hung/
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
