[jira] [Commented] (ARTEMIS-3509) STOMP Websocket client disconnected after ConcurrentModificationException in Broker
[
https://issues.apache.org/jira/browse/ARTEMIS-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17500844#comment-17500844
]
Luigi Leoni commented on ARTEMIS-3509:
--
Hi all, same issue here on 2.16.0,2.19.0,2.20.0
On a test using 2-3 clients stomp subscribed to a topic (with retroactive and
last value), randomly appears the same error reported by other users:
{code:java}
2022-03-03 10:51:55,299 WARN [org.apache.activemq.artemis.core.server]
AMQ222151: removing consumer which did not handle a message,
consumer=ServerConsumerImpl [id=1214893, filter=null, binding=LocalQueueBinding
[address=/topic/testTopic,
queue=QueueImpl[name=8f3257c4-9ad7-11ec-a207-005056a4b6a2,
postOffice=PostOfficeImpl
[server=ActiveMQServerImpl::serverUUID=2381ce67-9332-11ec-a207-005056a4b6a2],
temp=true]@14456cca, filter=null, name=8f3257c4-9ad7-11ec-a207-005056a4b6a2,
clusterName=8f3257c4-9ad7-11ec-a207-005056a4b6a22381ce67-9332-11ec-a207-005056a4b6a2]],
message=HolderReference@3b4125d8[ref=Reference[1106275]:RELIABLE:AMQPStandardMessage(
[durable=true, messageID=1106275,
address=$.artemis.internal./topic/testTopic.address.retro, size=1775,
applicationProperties={item_key=236c4829-85a5-4405-aa82-d225bf2b2efa,
timestamp=1646228931197},
properties=Properties{messageId=ID:6dd9b4e7-851e-4c5f-97b3-30ecd57e08a3:1:2:2-107,
userId=null, to='$.artemis.internal./topic/testTopic.address.retro',
subject='null', replyTo='null', correlationId=null, contentType=null,
contentEncoding=null, absoluteExpiryTime=Fri Mar 04 14:48:51 CET 2022,
creationTime=Wed Mar 02 14:48:51 CET 2022, groupId='null', groupSequence=null,
replyToGroupId='null'}, extraProperties =
TypedProperties[_AMQ_ORIG_QUEUE=$.artemis.internal./topic/testTopic.divert.retro,_AMQ_AD=$.artemis.internal./topic/testTopic.address.retro,_AMQ_ORIG_ADDRESS=/topic/testTopic,_AMQ_ORIG_MESSAGE_ID=1106274]]]:
java.util.ConcurrentModificationException
at
java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1043)
[java.base:]
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:997)
[java.base:]
at
org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:179)
at
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:579)
at
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$100(TransactionImpl.java:40){code}
Error random occurs, unpredictable and unable to find a way to reproduce.
When this exception raised, client stop receiving new messages but no
disconnection occurs, the same client continues receive messages from other
topic subscribed.
> STOMP Websocket client disconnected after ConcurrentModificationException in
> Broker
> ---
>
> Key: ARTEMIS-3509
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3509
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Affects Versions: 2.16.0
>Reporter: Elodie Vandromme
>Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Web clients connected through WebSocket are randomly disconnected from
> artemis broker without notification of the disconnection.
> A ConcurrentModificationException is raised;
> {noformat}
> [2021-09-28 10:26:53,591] - [WILDFLY] - [WARN ] - [Thread-11
> ([ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba|mailto:ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba])]
> - [org.apache.activemq.artemis.core.server] : *AMQ222151: removing consumer
> which did not handle a message*, consumer=ServerConsumerImpl [id=1990,
> filter=null, binding=LocalQueueBinding [address=jms.topic.defended_assetFull,
> queue=QueueImpl[name=d1586796-2035-11ec-8d2e-00505689a76b,
> postOffice=PostOfficeImpl
> [server=ActiveMQServerImpl::serverUUID=2f00c36f-2034-11ec-8d2e-00505689a76b],
> temp=true]@4180826b, filter=null, name=d1586796-2035-11ec-8d2e-00505689a76b,
> clusterName=d1586796-2035-11ec-8d2e-00505689a76b2f00c36f-2034-11ec-8d2e-00505689a76b]],
>
> message=Reference[2019]:RELIABLE:CoreMessage[messageID=2019,durable=true,userID=d5e2d1c3-2035-11ec-8d2e-00505689a76b,priority=4,
> timestamp=Tue Sep 28 10:26:53 CEST 2021,expiration=0, durable=true,
> address=jms.topic.defended_assetFull,size=6520,properties=TypedProperties[__AMQ_CID=f9eab79a-2034-11ec-8d2e-00505689a76b,_AMQ_ROUTING_TYPE=0]]@155890432:
> java.util.ConcurrentModificationException
> at
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
> at java.util.ArrayList$Itr.next(ArrayList.java:859)
> at
>
[jira] [Commented] (ARTEMIS-3509) STOMP Websocket client disconnected after ConcurrentModificationException in Broker
[
https://issues.apache.org/jira/browse/ARTEMIS-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17490964#comment-17490964
]
Benoît BERTHONNEAU commented on ARTEMIS-3509:
-
Hi all, I'm working with [~Elodie] and I think that we did find the issue:
everything is said in the stacktrace...
The list "{{refToAcks}}" is browsed inside a non synhronized method
"{{afterCommit()}}" and a concurrent access is raised in some (unknown)
circonstances. This is also the case in the method "{{afterRollback}}".
We are currently testing a patch (by adding a "{{synchronized}}" keyword on
these 2 methods...
> STOMP Websocket client disconnected after ConcurrentModificationException in
> Broker
> ---
>
> Key: ARTEMIS-3509
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3509
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Affects Versions: 2.16.0
>Reporter: Elodie Vandromme
>Priority: Major
>
> Web clients connected through WebSocket are randomly disconnected from
> artemis broker without notification of the disconnection.
> A ConcurrentModificationException is raised;
> {noformat}
> [2021-09-28 10:26:53,591] - [WILDFLY] - [WARN ] - [Thread-11
> ([ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba|mailto:ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba])]
> - [org.apache.activemq.artemis.core.server] : *AMQ222151: removing consumer
> which did not handle a message*, consumer=ServerConsumerImpl [id=1990,
> filter=null, binding=LocalQueueBinding [address=jms.topic.defended_assetFull,
> queue=QueueImpl[name=d1586796-2035-11ec-8d2e-00505689a76b,
> postOffice=PostOfficeImpl
> [server=ActiveMQServerImpl::serverUUID=2f00c36f-2034-11ec-8d2e-00505689a76b],
> temp=true]@4180826b, filter=null, name=d1586796-2035-11ec-8d2e-00505689a76b,
> clusterName=d1586796-2035-11ec-8d2e-00505689a76b2f00c36f-2034-11ec-8d2e-00505689a76b]],
>
> message=Reference[2019]:RELIABLE:CoreMessage[messageID=2019,durable=true,userID=d5e2d1c3-2035-11ec-8d2e-00505689a76b,priority=4,
> timestamp=Tue Sep 28 10:26:53 CEST 2021,expiration=0, durable=true,
> address=jms.topic.defended_assetFull,size=6520,properties=TypedProperties[__AMQ_CID=f9eab79a-2034-11ec-8d2e-00505689a76b,_AMQ_ROUTING_TYPE=0]]@155890432:
> java.util.ConcurrentModificationException
> at
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
> at java.util.ArrayList$Itr.next(ArrayList.java:859)
> at
> org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:179)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:579)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$100(TransactionImpl.java:40)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$2.done(TransactionImpl.java:322)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:190)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:128)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.afterCompleteOperations(AbstractJournalStorageManager.java:321)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:313)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:264)
> at
> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.commit(ServerSessionImpl.java:1284)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:164)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:116)
> at
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:511)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:3744)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3048)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2400(QueueImpl.java:125)
> at
>
[jira] [Commented] (ARTEMIS-3509) STOMP Websocket client disconnected after ConcurrentModificationException in Broker
[
https://issues.apache.org/jira/browse/ARTEMIS-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484837#comment-17484837
]
Justin Bertram commented on ARTEMIS-3509:
-
Any update here? Can you provide any of the details previously requested? Have
you tried upgrading to the latest release (i.e. 2.20.0)?
> STOMP Websocket client disconnected after ConcurrentModificationException in
> Broker
> ---
>
> Key: ARTEMIS-3509
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3509
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Affects Versions: 2.16.0
>Reporter: Elodie Vandromme
>Priority: Major
>
> Web clients connected through WebSocket are randomly disconnected from
> artemis broker without notification of the disconnection.
> A ConcurrentModificationException is raised;
> {noformat}
> [2021-09-28 10:26:53,591] - [WILDFLY] - [WARN ] - [Thread-11
> ([ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba|mailto:ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba])]
> - [org.apache.activemq.artemis.core.server] : *AMQ222151: removing consumer
> which did not handle a message*, consumer=ServerConsumerImpl [id=1990,
> filter=null, binding=LocalQueueBinding [address=jms.topic.defended_assetFull,
> queue=QueueImpl[name=d1586796-2035-11ec-8d2e-00505689a76b,
> postOffice=PostOfficeImpl
> [server=ActiveMQServerImpl::serverUUID=2f00c36f-2034-11ec-8d2e-00505689a76b],
> temp=true]@4180826b, filter=null, name=d1586796-2035-11ec-8d2e-00505689a76b,
> clusterName=d1586796-2035-11ec-8d2e-00505689a76b2f00c36f-2034-11ec-8d2e-00505689a76b]],
>
> message=Reference[2019]:RELIABLE:CoreMessage[messageID=2019,durable=true,userID=d5e2d1c3-2035-11ec-8d2e-00505689a76b,priority=4,
> timestamp=Tue Sep 28 10:26:53 CEST 2021,expiration=0, durable=true,
> address=jms.topic.defended_assetFull,size=6520,properties=TypedProperties[__AMQ_CID=f9eab79a-2034-11ec-8d2e-00505689a76b,_AMQ_ROUTING_TYPE=0]]@155890432:
> java.util.ConcurrentModificationException
> at
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
> at java.util.ArrayList$Itr.next(ArrayList.java:859)
> at
> org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:179)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:579)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$100(TransactionImpl.java:40)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$2.done(TransactionImpl.java:322)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:190)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:128)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.afterCompleteOperations(AbstractJournalStorageManager.java:321)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:313)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:264)
> at
> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.commit(ServerSessionImpl.java:1284)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:164)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:116)
> at
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:511)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:3744)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3048)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2400(QueueImpl.java:125)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4119)
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
> at
>
[jira] [Commented] (ARTEMIS-3509) STOMP Websocket client disconnected after ConcurrentModificationException in Broker
[
https://issues.apache.org/jira/browse/ARTEMIS-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457273#comment-17457273
]
Justin Bertram commented on ARTEMIS-3509:
-
In my experience it's relatively uncommon for logs to be very helpful in this
kind of scenario. Given that your application is exposing this exception it's
feasible that you could work backwards to create a test-case based on what your
application is doing. At the very least, a clear & complete description of what
the application is doing might help someone else reproduce the exception. Can
you provide either of these?
> STOMP Websocket client disconnected after ConcurrentModificationException in
> Broker
> ---
>
> Key: ARTEMIS-3509
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3509
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Affects Versions: 2.16.0
>Reporter: Elodie Vandromme
>Priority: Major
>
> Web clients connected through WebSocket are randomly disconnected from
> artemis broker without notification of the disconnection.
> A ConcurrentModificationException is raised;
> {noformat}
> [2021-09-28 10:26:53,591] - [WILDFLY] - [WARN ] - [Thread-11
> ([ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba|mailto:ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba])]
> - [org.apache.activemq.artemis.core.server] : *AMQ222151: removing consumer
> which did not handle a message*, consumer=ServerConsumerImpl [id=1990,
> filter=null, binding=LocalQueueBinding [address=jms.topic.defended_assetFull,
> queue=QueueImpl[name=d1586796-2035-11ec-8d2e-00505689a76b,
> postOffice=PostOfficeImpl
> [server=ActiveMQServerImpl::serverUUID=2f00c36f-2034-11ec-8d2e-00505689a76b],
> temp=true]@4180826b, filter=null, name=d1586796-2035-11ec-8d2e-00505689a76b,
> clusterName=d1586796-2035-11ec-8d2e-00505689a76b2f00c36f-2034-11ec-8d2e-00505689a76b]],
>
> message=Reference[2019]:RELIABLE:CoreMessage[messageID=2019,durable=true,userID=d5e2d1c3-2035-11ec-8d2e-00505689a76b,priority=4,
> timestamp=Tue Sep 28 10:26:53 CEST 2021,expiration=0, durable=true,
> address=jms.topic.defended_assetFull,size=6520,properties=TypedProperties[__AMQ_CID=f9eab79a-2034-11ec-8d2e-00505689a76b,_AMQ_ROUTING_TYPE=0]]@155890432:
> java.util.ConcurrentModificationException
> at
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
> at java.util.ArrayList$Itr.next(ArrayList.java:859)
> at
> org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:179)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:579)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$100(TransactionImpl.java:40)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$2.done(TransactionImpl.java:322)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:190)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:128)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.afterCompleteOperations(AbstractJournalStorageManager.java:321)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:313)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:264)
> at
> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.commit(ServerSessionImpl.java:1284)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:164)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:116)
> at
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:511)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:3744)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3048)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2400(QueueImpl.java:125)
> at
>
[jira] [Commented] (ARTEMIS-3509) STOMP Websocket client disconnected after ConcurrentModificationException in Broker
[
https://issues.apache.org/jira/browse/ARTEMIS-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17457246#comment-17457246
]
Elodie Vandromme commented on ARTEMIS-3509:
---
[~jbertram], we reproduced it randomly with our application that I can not
provide to you. And as I do not know the root cause of that issue, I'm not able
to provide a sample that is able to reproduce the issue neither.
We could provide more logs if this can be helful for you (maybe you could
provide us the list of logs to activate or a jar with more relevant logs that
could be useful to you).
> STOMP Websocket client disconnected after ConcurrentModificationException in
> Broker
> ---
>
> Key: ARTEMIS-3509
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3509
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Affects Versions: 2.16.0
>Reporter: Elodie Vandromme
>Priority: Major
>
> Web clients connected through WebSocket are randomly disconnected from
> artemis broker without notification of the disconnection.
> A ConcurrentModificationException is raised;
> {noformat}
> [2021-09-28 10:26:53,591] - [WILDFLY] - [WARN ] - [Thread-11
> ([ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba|mailto:ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba])]
> - [org.apache.activemq.artemis.core.server] : *AMQ222151: removing consumer
> which did not handle a message*, consumer=ServerConsumerImpl [id=1990,
> filter=null, binding=LocalQueueBinding [address=jms.topic.defended_assetFull,
> queue=QueueImpl[name=d1586796-2035-11ec-8d2e-00505689a76b,
> postOffice=PostOfficeImpl
> [server=ActiveMQServerImpl::serverUUID=2f00c36f-2034-11ec-8d2e-00505689a76b],
> temp=true]@4180826b, filter=null, name=d1586796-2035-11ec-8d2e-00505689a76b,
> clusterName=d1586796-2035-11ec-8d2e-00505689a76b2f00c36f-2034-11ec-8d2e-00505689a76b]],
>
> message=Reference[2019]:RELIABLE:CoreMessage[messageID=2019,durable=true,userID=d5e2d1c3-2035-11ec-8d2e-00505689a76b,priority=4,
> timestamp=Tue Sep 28 10:26:53 CEST 2021,expiration=0, durable=true,
> address=jms.topic.defended_assetFull,size=6520,properties=TypedProperties[__AMQ_CID=f9eab79a-2034-11ec-8d2e-00505689a76b,_AMQ_ROUTING_TYPE=0]]@155890432:
> java.util.ConcurrentModificationException
> at
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
> at java.util.ArrayList$Itr.next(ArrayList.java:859)
> at
> org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:179)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:579)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$100(TransactionImpl.java:40)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$2.done(TransactionImpl.java:322)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:190)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:128)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.afterCompleteOperations(AbstractJournalStorageManager.java:321)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:313)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:264)
> at
> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.commit(ServerSessionImpl.java:1284)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:164)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:116)
> at
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:511)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:3744)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3048)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2400(QueueImpl.java:125)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4119)
>
[jira] [Commented] (ARTEMIS-3509) STOMP Websocket client disconnected after ConcurrentModificationException in Broker
[
https://issues.apache.org/jira/browse/ARTEMIS-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455368#comment-17455368
]
Justin Bertram commented on ARTEMIS-3509:
-
[~Elodie], can you elaborate on how you were able to reproduce the exception? I
need sufficient detail to reproduce the failure for myself. Thanks!
> STOMP Websocket client disconnected after ConcurrentModificationException in
> Broker
> ---
>
> Key: ARTEMIS-3509
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3509
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Affects Versions: 2.16.0
>Reporter: Elodie Vandromme
>Priority: Major
>
> Web clients connected through WebSocket are randomly disconnected from
> artemis broker without notification of the disconnection.
> A ConcurrentModificationException is raised;
> {noformat}
> [2021-09-28 10:26:53,591] - [WILDFLY] - [WARN ] - [Thread-11
> ([ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba|mailto:ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba])]
> - [org.apache.activemq.artemis.core.server] : *AMQ222151: removing consumer
> which did not handle a message*, consumer=ServerConsumerImpl [id=1990,
> filter=null, binding=LocalQueueBinding [address=jms.topic.defended_assetFull,
> queue=QueueImpl[name=d1586796-2035-11ec-8d2e-00505689a76b,
> postOffice=PostOfficeImpl
> [server=ActiveMQServerImpl::serverUUID=2f00c36f-2034-11ec-8d2e-00505689a76b],
> temp=true]@4180826b, filter=null, name=d1586796-2035-11ec-8d2e-00505689a76b,
> clusterName=d1586796-2035-11ec-8d2e-00505689a76b2f00c36f-2034-11ec-8d2e-00505689a76b]],
>
> message=Reference[2019]:RELIABLE:CoreMessage[messageID=2019,durable=true,userID=d5e2d1c3-2035-11ec-8d2e-00505689a76b,priority=4,
> timestamp=Tue Sep 28 10:26:53 CEST 2021,expiration=0, durable=true,
> address=jms.topic.defended_assetFull,size=6520,properties=TypedProperties[__AMQ_CID=f9eab79a-2034-11ec-8d2e-00505689a76b,_AMQ_ROUTING_TYPE=0]]@155890432:
> java.util.ConcurrentModificationException
> at
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
> at java.util.ArrayList$Itr.next(ArrayList.java:859)
> at
> org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:179)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:579)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$100(TransactionImpl.java:40)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$2.done(TransactionImpl.java:322)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:190)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:128)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.afterCompleteOperations(AbstractJournalStorageManager.java:321)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:313)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:264)
> at
> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.commit(ServerSessionImpl.java:1284)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:164)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:116)
> at
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:511)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:3744)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3048)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2400(QueueImpl.java:125)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4119)
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
> at
> org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
> at
>
[jira] [Commented] (ARTEMIS-3509) STOMP Websocket client disconnected after ConcurrentModificationException in Broker
[
https://issues.apache.org/jira/browse/ARTEMIS-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17455030#comment-17455030
]
Elodie Vandromme commented on ARTEMIS-3509:
---
Tested on 2.19.0 version of Artemis in standalone mode with the same result :
{noformat}
2021-12-07 15:08:50,017 WARN [org.apache.activemq.artemis.core.server]
AMQ222151: removing consumer which did not handle a message,
consumer=ServerConsumerImpl [id=2147502803, filter=null,
binding=LocalQueueBinding [address=jms.topic.acmFull,
queue=QueueImpl[name=26d11fd1-5767-11ec-bc7c-3024a9823a44,
postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=127.0.0.1],
temp=true]@44b4401b, filter=null, name=26d11fd1-5767-11ec-bc7c-3024a9823a44,
clusterName=26d11fd1-5767-11ec-bc7c-3024a9823a4415fd2ebf-5735-11ec-828b-3024a9823a44]],
message=Reference[2147502950]:RELIABLE:CoreMessage[messageID=2147502950,durable=true,userID=338e89a1-5767-11ec-b3a8-3024a9823a44,priority=4,
timestamp=Tue Dec 07 15:08:49 CET 2021,expiration=0, durable=true,
address=jms.topic.acmFull,size=3664,properties=TypedProperties[__AMQ_CID=fe63a897-5765-11ec-b3a8-3024a9823a44,_AMQ_ROUTING_TYPE=0]]@2056760686:
java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
[rt.jar:1.8.0_222-4-redhat]
at java.util.ArrayList$Itr.next(ArrayList.java:859)
[rt.jar:1.8.0_222-4-redhat]
at
org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:174)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:564)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$2.done(TransactionImpl.java:305)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:195)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:141)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.afterCompleteOperations(AbstractJournalStorageManager.java:329)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:296)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:247)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.commit(ServerSessionImpl.java:1320)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:164)
[artemis-stomp-protocol-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:116)
[artemis-stomp-protocol-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:518)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:3840)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3142)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2400(QueueImpl.java:129)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4207)
[artemis-server-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
[artemis-commons-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
[artemis-commons-2.19.0.jar:2.19.0]
at
org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65)
[artemis-commons-2.19.0.jar:2.19.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[rt.jar:1.8.0_222-4-redhat]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[rt.jar:1.8.0_222-4-redhat]
at
org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
[artemis-commons-2.19.0.jar:2.19.0]{noformat}
> STOMP Websocket client disconnected after ConcurrentModificationException in
> Broker
> ---
>
> Key: ARTEMIS-3509
> URL:
[jira] [Commented] (ARTEMIS-3509) STOMP Websocket client disconnected after ConcurrentModificationException in Broker
[
https://issues.apache.org/jira/browse/ARTEMIS-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17441303#comment-17441303
]
Clebert Suconic commented on ARTEMIS-3509:
--
The best for you would be to reproduce the issue using a self contained example.
multiple producers, multiple consumers using the same semantics you use to send
and consume.
Usually those tests would reproduce the issue much faster than what you see in
a production system. Then you would be able to alternate between 2.16 and 2.19
and verify if the issue is fixed or not. We would at that point have a way to
debug your issue.
> STOMP Websocket client disconnected after ConcurrentModificationException in
> Broker
> ---
>
> Key: ARTEMIS-3509
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3509
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Affects Versions: 2.16.0
>Reporter: Elodie Vandromme
>Priority: Major
>
> Web clients connected through WebSocket are randomly disconnected from
> artemis broker without notification of the disconnection.
> A ConcurrentModificationException is raised;
> {noformat}
> [2021-09-28 10:26:53,591] - [WILDFLY] - [WARN ] - [Thread-11
> ([ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba|mailto:ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba])]
> - [org.apache.activemq.artemis.core.server] : *AMQ222151: removing consumer
> which did not handle a message*, consumer=ServerConsumerImpl [id=1990,
> filter=null, binding=LocalQueueBinding [address=jms.topic.defended_assetFull,
> queue=QueueImpl[name=d1586796-2035-11ec-8d2e-00505689a76b,
> postOffice=PostOfficeImpl
> [server=ActiveMQServerImpl::serverUUID=2f00c36f-2034-11ec-8d2e-00505689a76b],
> temp=true]@4180826b, filter=null, name=d1586796-2035-11ec-8d2e-00505689a76b,
> clusterName=d1586796-2035-11ec-8d2e-00505689a76b2f00c36f-2034-11ec-8d2e-00505689a76b]],
>
> message=Reference[2019]:RELIABLE:CoreMessage[messageID=2019,durable=true,userID=d5e2d1c3-2035-11ec-8d2e-00505689a76b,priority=4,
> timestamp=Tue Sep 28 10:26:53 CEST 2021,expiration=0, durable=true,
> address=jms.topic.defended_assetFull,size=6520,properties=TypedProperties[__AMQ_CID=f9eab79a-2034-11ec-8d2e-00505689a76b,_AMQ_ROUTING_TYPE=0]]@155890432:
> java.util.ConcurrentModificationException
> at
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
> at java.util.ArrayList$Itr.next(ArrayList.java:859)
> at
> org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:179)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:579)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$100(TransactionImpl.java:40)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$2.done(TransactionImpl.java:322)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:190)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:128)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.afterCompleteOperations(AbstractJournalStorageManager.java:321)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:313)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:264)
> at
> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.commit(ServerSessionImpl.java:1284)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:164)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:116)
> at
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:511)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:3744)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3048)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2400(QueueImpl.java:125)
> at
>
[jira] [Commented] (ARTEMIS-3509) STOMP Websocket client disconnected after ConcurrentModificationException in Broker
[
https://issues.apache.org/jira/browse/ARTEMIS-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440703#comment-17440703
]
Justin Bertram commented on ARTEMIS-3509:
-
The 2.16.0 release is a bit old at this point. Can you reproduce with the
latest 2.20.0 release?
In general, I would recommend running your broker _standalone_ (i.e. outside of
containers like WildFly). WildFly does, of course, integrate ActiveMQ Artemis
as its JMS implementation, but this integration makes upgrading the broker
difficult which can be important when you need bug fixes.
> STOMP Websocket client disconnected after ConcurrentModificationException in
> Broker
> ---
>
> Key: ARTEMIS-3509
> URL: https://issues.apache.org/jira/browse/ARTEMIS-3509
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
>Affects Versions: 2.16.0
>Reporter: Elodie Vandromme
>Priority: Major
>
> Web clients connected through WebSocket are randomly disconnected from
> artemis broker without notification of the disconnection.
> A ConcurrentModificationException is raised;
> {noformat}
> [2021-09-28 10:26:53,591] - [WILDFLY] - [WARN ] - [Thread-11
> ([ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba|mailto:ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba])]
> - [org.apache.activemq.artemis.core.server] : *AMQ222151: removing consumer
> which did not handle a message*, consumer=ServerConsumerImpl [id=1990,
> filter=null, binding=LocalQueueBinding [address=jms.topic.defended_assetFull,
> queue=QueueImpl[name=d1586796-2035-11ec-8d2e-00505689a76b,
> postOffice=PostOfficeImpl
> [server=ActiveMQServerImpl::serverUUID=2f00c36f-2034-11ec-8d2e-00505689a76b],
> temp=true]@4180826b, filter=null, name=d1586796-2035-11ec-8d2e-00505689a76b,
> clusterName=d1586796-2035-11ec-8d2e-00505689a76b2f00c36f-2034-11ec-8d2e-00505689a76b]],
>
> message=Reference[2019]:RELIABLE:CoreMessage[messageID=2019,durable=true,userID=d5e2d1c3-2035-11ec-8d2e-00505689a76b,priority=4,
> timestamp=Tue Sep 28 10:26:53 CEST 2021,expiration=0, durable=true,
> address=jms.topic.defended_assetFull,size=6520,properties=TypedProperties[__AMQ_CID=f9eab79a-2034-11ec-8d2e-00505689a76b,_AMQ_ROUTING_TYPE=0]]@155890432:
> java.util.ConcurrentModificationException
> at
> java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
> at java.util.ArrayList$Itr.next(ArrayList.java:859)
> at
> org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:179)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:579)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$100(TransactionImpl.java:40)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$2.done(TransactionImpl.java:322)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:190)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:128)
> at
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.afterCompleteOperations(AbstractJournalStorageManager.java:321)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:313)
> at
> org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:264)
> at
> org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.commit(ServerSessionImpl.java:1284)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:164)
> at
> org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:116)
> at
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:511)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:3744)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3048)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2400(QueueImpl.java:125)
> at
> org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4119)
> at
>
