GitHub user wy96f opened a pull request:
https://github.com/apache/activemq-artemis/pull/2476
Fix deadlock while getting queue message count during cleanup
ARTEMIS-2123 introduced a deadlock bug.
jstack shows:
Thread 1:
at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
at
org.apache.activemq.artemis.core.paging.impl.PagingStoreImpl.startPaging(PagingStoreImpl.java:481)
at
org.apache.activemq.artemis.core.paging.impl.PagingStoreImpl.addSize(PagingStoreImpl.java:739)
at
org.apache.activemq.artemis.core.paging.impl.PagingStoreImpl.nonDurableUp(PagingStoreImpl.java:952)
at
org.apache.activemq.artemis.api.core.RefCountMessage.incrementRefCount(RefCountMessage.java:50)
at
org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl.incrementDelayDeletionCount(LargeServerMessageImpl.java:149)
locked
org.apache.activemq.artemis.core.persistence.impl.journal.LargeServerMessageImpl@67359741
at
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl$LargeMessageDeliverer.<init>(ServerConsumerImpl.java:1171)
at
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl$LargeMessageDeliverer.<init>(ServerConsumerImpl.java:1151)
at
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.handle(ServerConsumerImpl.java:431)
locked java.lang.Object@3f31a7b3
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.handle(QueueImpl.java:2809)
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:2196)
locked org.apache.activemq.artemis.core.server.impl.QueueImpl@6c2bd0dc
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.access$1900(QueueImpl.java:105)
at
org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:3001)
locked
org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner@79dea1f9
Thread 2:
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.getScheduledCount(QueueImpl.java:1085)
blocked on org.apache.activemq.artemis.core.server.impl.QueueImpl@742b7e17
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.getMessageCount(QueueImpl.java:1077)
at
org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.deliverIfNecessary(PageCursorProviderImpl.java:610)
at
org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.cleanup(PageCursorProviderImpl.java:365)
locked
org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl@5aa5010
at
org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl$1.run(PageCursorProviderImpl.java:288)
The cleanup thread held pagingStore lock and requested queue lock. The
largeMessageDeliver held queue lock and requested pagingStore lock. Deadlock
occurred.
Put queue::getMessageCount outside of pagingstore lock to fix the bug.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wy96f/activemq-artemis
fix_deadlock_caused_by_getmessagecount
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/activemq-artemis/pull/2476.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2476
----
commit f3f63fc961ea9573a4b7caaf1fe19cec15fcac3a
Author: yang wei <wy96fyw@...>
Date: 2018-12-21T04:29:04Z
Fix deadlock while getting queue message count during cleanup
----
---
