[
https://issues.apache.org/jira/browse/ARTEMIS-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Frederik Petersen updated ARTEMIS-2276:
---------------------------------------
Description:
Hi,
we just experienced another deadlock in artemis.
I am attaching the thread dump. The issue seems to be here:
{code:java}
"Thread-0 (activemq-netty-threads)" Id=43 BLOCKED on
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
owned by "Thread-7
(ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)"
Id=87
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1332)
- blocked on
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1328)
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.acknowledge(QueueImpl.java:1295)
at
org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:207)
at
org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:199)
at
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.acknowledge(ServerConsumerImpl.java:874)
- locked
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl@31c578da
at
org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.acknowledge(ServerSessionImpl.java:891)
at
org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:159)
at
org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:111)
at
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:469)
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:2936)
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.deliverDirect(QueueImpl.java:2921)
- locked org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
[...]
"Thread-7
(ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)"
Id=87 BLOCKED on
org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217 owned by
"Thread-0 (activemq-netty-threads)" Id=43
at
org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:165)
- blocked on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
at
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:546)
- locked
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
[...]
{code}
Thread-0 locks QueueImpl@6b7ed217.
Thread-7 locks TransactionImpl@196f00e1.
Thread-0 waits for TransactionImpl@196f00e1.
Thread-7 waits for QueueImpl@6b7ed217.
We had a few mqtt producers, stomp consumers and stomp producers. A few hundred
items/sec. deliverDirect disabled on mqtt as suggested by [~clebertsuconic]
here:
[https://github.com/apache/activemq-artemis/pull/2577#issuecomment-472070556]
But I guess this dead lock is not the same as the one discussed there.
Thanks for your help.
was:
Hi,
we just experienced another deadlock in artemis.
I am attaching the thread dump. The issue seems to be here:
{code:java}
"Thread-0 (activemq-netty-threads)" Id=43 BLOCKED on
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
owned by "Thread-7
(ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)"
Id=87
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1332)
- blocked on
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1328)
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.acknowledge(QueueImpl.java:1295)
at
org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:207)
at
org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:199)
at
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.acknowledge(ServerConsumerImpl.java:874)
- locked
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl@31c578da
at
org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.acknowledge(ServerSessionImpl.java:891)
at
org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:159)
at
org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:111)
at
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:469)
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:2936)
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.deliverDirect(QueueImpl.java:2921)
- locked org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
[...]
"Thread-7
(ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)"
Id=87 BLOCKED on
org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217 owned by
"Thread-0 (activemq-netty-threads)" Id=43
at
org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:165)
- blocked on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
at
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:546)
- locked
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
[...]
{code}
Thread-0 locks QueueImpl@6b7ed217.
Thread-7 locks TransactionImpl@196f00e1.
Thread-0 waits for TransactionImpl@196f00e1.
Thread-7 waits for QueueImpl@6b7ed217.
We had a few mqtt producers, stomp consumers and stomp producers. A few hundred
items/sec. deliverDirect disabled on mqtt as suggested by [~clebertsuconic]
here:
[https://github.com/apache/activemq-artemis/pull/2577#issuecomment-472070556]
But I guess this dead lockis not the same as the one discussed there.
Thanks for your help.
> Deadlock between QueueImpl and TransactionImpl
> ----------------------------------------------
>
> Key: ARTEMIS-2276
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2276
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
> Affects Versions: 2.6.4
> Reporter: Frederik Petersen
> Priority: Major
> Attachments: artemis_thread_dump_2.log
>
>
> Hi,
> we just experienced another deadlock in artemis.
> I am attaching the thread dump. The issue seems to be here:
>
> {code:java}
> "Thread-0 (activemq-netty-threads)" Id=43 BLOCKED on
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
> owned by "Thread-7
> (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)"
> Id=87
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1332)
> - blocked on
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1328)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.acknowledge(QueueImpl.java:1295)
> at
> org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:207)
> at
> org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:199)
> at
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.acknowledge(ServerConsumerImpl.java:874)
> - locked
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl@31c578da
> at
> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.acknowledge(ServerSessionImpl.java:891)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:159)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:111)
> at
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:469)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:2936)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.deliverDirect(QueueImpl.java:2921)
> - locked org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
> [...]
> "Thread-7
> (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)"
> Id=87 BLOCKED on
> org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217 owned by
> "Thread-0 (activemq-netty-threads)" Id=43
> at
> org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:165)
> - blocked on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:546)
> - locked
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
> [...]
> {code}
>
> Thread-0 locks QueueImpl@6b7ed217.
> Thread-7 locks TransactionImpl@196f00e1.
> Thread-0 waits for TransactionImpl@196f00e1.
> Thread-7 waits for QueueImpl@6b7ed217.
> We had a few mqtt producers, stomp consumers and stomp producers. A few
> hundred items/sec. deliverDirect disabled on mqtt as suggested by
> [~clebertsuconic] here:
> [https://github.com/apache/activemq-artemis/pull/2577#issuecomment-472070556]
> But I guess this dead lock is not the same as the one discussed there.
> Thanks for your help.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
