[
https://issues.apache.org/jira/browse/AMQ-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
subhajit maiti updated AMQ-7066:
--------------------------------
Description:
Recently we observed a strange problem in one of our flows. We could see some
messages being stuck in the queue when we are using the message grouping and
dynamic scaling feature together.
*Usecase:*
We want to process the messages belonging to a user (application user) in a
sequence. Multiple users can send messages to the same queue, but messages
originating from transactions made by a user should be processed sequentially.
In order to do so we are using the "JMSXGroupID" header to group the messages
and the "username" serves as the group key.
*Our configuration and versions:*
# We are using ActiveMQ 5.15.5
# We are using camel routing to connect to ActiveMQ, both on the producer and
consumer side. Camel version : 2.20.1
# We are using the following properties in the consumer URI to enable dynamic
scaling : maxConcurrentConsumers, idleTaskExecutionLimit (defaulted to 10),
maxMessagesPerTask (defaulted to 250)
# JDK version : 1.8.0_91
*Observations:*
# The messages are stuck for some of the groups. Messages belonging to other
groups are processing fine. There is no identifiable pattern for the groups
that are stuck. During each execution some random groups got stuck.
# The messages get processed when the consumer container is restarted and
works fine for sometime before the issue recurs.
# When we verified from the admin console, we could see that the number of
consumers are going beyond the max value set for it.
# We verified from the thread dumps that all the consumers are healthy and are
in TIMED_WAITING state. Even though there are some messages stuck in the queue,
they are not being delivered to the waiting consumers.
# We have entry-exit logs for the producers and consumers. We could see the
messages are being sent successfully by the producer and are never delivered to
the consumer.
*Expected behaviour* : All messages should be delivered to the consumers.
*Actual behaviour* : Some of the messages belonging to random groups get stuck
in the queue until the consumer container is restarted.
was:
Recently we observed a strange problem in one of our flows. We could see some
messages being stuck in the queue when we are using the message grouping and
dynamic scaling feature together.
*Usecase:*
We want to process the messages belonging to a user (application user) in a
sequence. Multiple users can send messages to the same queue, but messages
originating from transactions made by a user should be processed sequentially.
In order to do so we are using the "JMSXGroupID" header to group the messages
and the "username" serves as the group key.
*Our configuration and versions:*
# We are using ActiveMQ 5.15.5
# We are using camel routing to connect to ActiveMQ, both on the producer and
consumer side. Camel version : 2.20.1
# We are using the following properties in the consumer URI to enable dynamic
scaling : maxConcurrentConsumers,
idleTaskExecutionLimit (defaulted to 10)
maxMessagesPerTask (defaulted to 250)
# JDK version : 1.8.0_91
*Observations:*
# The messages are stuck for some of the groups. Messages belonging to other
groups are processing fine. There is no identifiable pattern for the groups
that are stuck. During each execution some random groups got stuck.
# The messages get processed when the consumer container is restarted and
works fine for sometime before the issue recurs.
# When we verified from the admin console, we could see that the number of
consumers are going beyond the max value set for it.
# We verified from the thread dumps that all the consumers are healthy and are
in TIMED_WAITING state. Even though there are some messages stuck in the queue,
they are not being delivered to the waiting consumers.
# We have entry-exit logs for the producers and consumers. We could see the
messages are being sent successfully by the producer and are never delivered to
the consumer.
Expected behaviour : All messages should be delivered to the consumers.
Actual behaviour : Some of the messages belonging to random groups get stuck in
the queue until the consumer container is restarted.
> Using dynamic scaling and message grouping feature together makes messages
> stuck
> --------------------------------------------------------------------------------
>
> Key: AMQ-7066
> URL: https://issues.apache.org/jira/browse/AMQ-7066
> Project: ActiveMQ
> Issue Type: Bug
> Components: activemq-camel
> Affects Versions: 5.15.5
> Reporter: subhajit maiti
> Priority: Critical
>
> Recently we observed a strange problem in one of our flows. We could see some
> messages being stuck in the queue when we are using the message grouping and
> dynamic scaling feature together.
> *Usecase:*
> We want to process the messages belonging to a user (application user) in a
> sequence. Multiple users can send messages to the same queue, but messages
> originating from transactions made by a user should be processed
> sequentially. In order to do so we are using the "JMSXGroupID" header to
> group the messages and the "username" serves as the group key.
> *Our configuration and versions:*
> # We are using ActiveMQ 5.15.5
> # We are using camel routing to connect to ActiveMQ, both on the producer
> and consumer side. Camel version : 2.20.1
> # We are using the following properties in the consumer URI to enable
> dynamic scaling : maxConcurrentConsumers, idleTaskExecutionLimit (defaulted
> to 10), maxMessagesPerTask (defaulted to 250)
> # JDK version : 1.8.0_91
> *Observations:*
> # The messages are stuck for some of the groups. Messages belonging to other
> groups are processing fine. There is no identifiable pattern for the groups
> that are stuck. During each execution some random groups got stuck.
> # The messages get processed when the consumer container is restarted and
> works fine for sometime before the issue recurs.
> # When we verified from the admin console, we could see that the number of
> consumers are going beyond the max value set for it.
> # We verified from the thread dumps that all the consumers are healthy and
> are in TIMED_WAITING state. Even though there are some messages stuck in the
> queue, they are not being delivered to the waiting consumers.
> # We have entry-exit logs for the producers and consumers. We could see the
> messages are being sent successfully by the producer and are never delivered
> to the consumer.
> *Expected behaviour* : All messages should be delivered to the consumers.
> *Actual behaviour* : Some of the messages belonging to random groups get
> stuck in the queue until the consumer container is restarted.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
