[
https://issues.apache.org/jira/browse/AMQ-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494330#comment-13494330
]
Gary Tully commented on AMQ-4166:
---------------------------------
I think the processing of expired messages in error is the root cause or the
deadlock.
The correct expiry check is in http://svn.apache.org/viewvc?rev=1407640&view=rev
There is only one valid path to scheduled redelivery and this is through the
receipt of a poison ack with just a subscription dispatch lock held.
If optimizedDispatch is used, such that the scheduler send thread does a
dispatch, there is a deadlock possibility.
The fix is either to split schedule and execute in the scheduler or let the
redelivery plugin do the schedule async. The simplest solution is to schedule
async i think.
@Sergiy, from your config, you don't use optimizedDispatch so you should be set.
Would it be possible to validate using a 5.8-SNAPSHOT.
> RedeliveryPlugin causes a deadlock with JobSchedulerImpl
> --------------------------------------------------------
>
> Key: AMQ-4166
> URL: https://issues.apache.org/jira/browse/AMQ-4166
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.7.0
> Environment: Reproduced on Windows 8, Windows Vista, MacOS X
> with Oracle jdk 1.7.0_07. ActiveMQ is started embedded using RAR inside
> Glassfish 3.1.2.2.
> Reporter: Sergiy Barlabanov
> Attachments: broker-config.xml, stack-trace-1.txt, stack-trace-2.txt
>
>
> Originates from the forum discussion
> http://activemq.2283324.n4.nabble.com/RedeliveryPlugin-causes-a-deadlock-with-JobSchedulerImpl-in-ActiveMQ-5-7-0-tt4659019.html
> we have RedeliveryPlugin causing thread deadlock together with
> JobSchedulerImpl. ActiveMQ version is 5.7.0. We activated RedeliveryPlugin in
> our broker config xml (see below). There two stacktraces below as well. One
> is from ActiveMQ VMTransport thread, which tries to send a message to a dead
> letter queue using RedeliveryPlugin. RedeliveryPlugin just tries to
> reschedule the message for redelivery and for that it calls JobSchedulerImpl
> and blocks on its synchronized method "schedule". On the way "consumersLock"
> is locked.
> Another stack trace is from JobScheduler:JMS thread, which fires a job to
> redeliver some message and tries to send it using the same queue used by the
> VMTransport thread. And it blocks on that consumersLock locked by the
> VMTransport thread. And this occurs in JobSchedulerImpl#mainLoop method
> inside synchronized {} block causing a deadlock, since the VMTransport thread
> tries to call another synchronized method of JobSchedulerImpl. The art how
> RedeliveryPlugin and JobSchedulerImpl are programmed seems to be quite
> dangerous, since they both access the queues and try to acquire queue locks.
> And additionally synchronized methods of JobSchedulerImpl are called directly
> from RedeliveryPlugin making that to a nice source of thread deadlocks. And I
> see no measures taken in the code to avoid these deadlocks.
> We can reproduce it quite often if we start ActiveMQ with empty stores
> (kahadb and scheduler stores are deleted manually from the file system before
> startup). But looking at the code, I would say that the problem may occur in
> any situation in any deployment scenario (standalone or embedded in a JEE
> container). It is just enough to have some Transport thread redelivering a
> message and the JobScheduler thread trying to fire a job at the same moment
> on the same queue.
> And another strange thing, which is may be has nothing to do with the
> deadlock but is still strange, is that according to the stack trace
> RedeliveryPlugin tries to redeliver an expired message.
> broker config and the stack traces are attached to the issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
