[
https://issues.apache.org/jira/browse/ARTEMIS-5481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952538#comment-17952538
]
Marcel Bil edited comment on ARTEMIS-5481 at 5/19/25 8:11 AM:
--------------------------------------------------------------
I'll had some indepth investigation and will add some history. We were on
version 2.31.2. Upgrading to 2.37 influenced different behaviour regarding
management queues. Eventually out of connections. Out of memory. Especially on
environments with more consumers this is a problem. Below is a good example of
the effect. Same configuration. Same amount of consumers (application/services
landscape isn't changed). A stable pattern of funcitonal messages (mostly
AMQ224113) on the left (Artemis 2.31.2.), in the center the effect of upgrading
to 2.37 with it's issues and on the right the rollback to 2.3.1.2.
!image-2025-05-19-10-03-28-268.png!
And a second screen where you see stability on the left and right (2.31). And
the 'stairs' are in the center. Gaps are crashes. Stopping of the service. Also
effected by the Linux oom-killer. As you can see the amount of consumers stays
the same but the amount of connection is majorly increased. What also is
interesting is that I performed a tcpdump on both versions and the amount of
packets per unique host hasn't changed. You see a ~50% increase of connections
in the graphs but that's not visible on the network interface:
!cjiblx8408_2.31_2.37_ATO_CL1.PNG!
We then tried other versions from new to old. 2.35 shows the same behaviour as
2.31.2. From version 2.36 we see the change in behaviour. We see the affect but
can't seem to find the cause in reported issues/fixes or code. Please let me
know if you need additional information.
was (Author: JIRAUSER309737):
I'll had some indepth investigation and will add some history. We were on
version 2.31.2. Upgrading to 2.37 influenced different behaviour regarding
management queues. Eventually out of connections. Out of memory. Especially on
environments with more consumers this is a problem. Below is a good example of
the effect. Same configuration. Same amount of consumers (application/services
landscape isn't changed). A stable pattern of funcitonal messages (mostly
AMQ224113) on the left (Artemis 2.31.2.), in the center the effect of upgrading
to 2.37 with it's issues and on the right the rollback to 2.3.1.2.
!image-2025-05-19-10-03-28-268.png!
And a second screen where you see stability on the left and right (2.31). And
the 'stairs' are in the center. Gaps are crashes. Stopping of the service. Also
effected by the Linux oom-killer. As you can see the amount of consumers stays
the same but the amount of connection is majorly increased. What also is
interesting is that I performed a tcpdump on both versions and the amount of
packets per unique host hasn't changed. You see a ~50% increase of connections
in the graphs but that's not visible on the network interface:
!cjiblx8408_2.31_2.37_ATO_CL1.PNG!
> Accumulation activemq.managment queues and addresses, increase connections
> --------------------------------------------------------------------------
>
> Key: ARTEMIS-5481
> URL: https://issues.apache.org/jira/browse/ARTEMIS-5481
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
> Affects Versions: 2.36.0, 2.37.0, 2.38.0, 2.39.0, 2.40.0, 2.41.0
> Reporter: Diederick
> Assignee: Clebert Suconic
> Priority: Major
> Labels: pull-request-available
> Attachments: 8408_2.31_2.37_Elastic.PNG,
> cjiblx8408_2.31_2.37_ATO_CL1.PNG, image-2025-05-15-13-14-20-461.png,
> image-2025-05-19-10-03-28-268.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We have a lot of problems with Artemis version 2.36.0 and higher (in a HA
> setup with shared file system). Our clients use activemq.management requests
> to see which node is active(primary) etc.
> Now from version 2.36.0 and higher there is a accumulation of these
> activemq.management queues and addresses. There is also a big increase in
> connections (amount of client connections is the same). This accumulation
> eats all the resources on the node (cpu and memory). Eventually the node give
> first a out of memory on the GUI and then node it self becomes unresponsive
> and dies (OOM kill) (no failover).
> As you can see in this graphic below (the green line), for version 2.36.0,
> there is a increase of connections (around 800). The behaviour is that
> management queues address slowly accumulate and eating resources (cpu and
> memory)
> After the gap you see version 2.35.0, which is a 'normal' amount of
> connections. Also the accumulation of management addresses and queues are
> cleaned up with default parameters
> !image-2025-05-15-13-14-20-461.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@activemq.apache.org
For additional commands, e-mail: issues-h...@activemq.apache.org
For further information, visit: https://activemq.apache.org/contact
