OutOfMemory provoked by Advisory messages
-----------------------------------------
Key: AMQ-2533
URL: https://issues.apache.org/activemq/browse/AMQ-2533
Project: ActiveMQ
Issue Type: Bug
Components: Broker
Affects Versions: 5.2.0
Reporter: Philippe Mouawad
Priority: Critical
Attachments: Image 11.png
Hello,
We are in an environment where we have 1 Broker and 6 clients listener to one
Topic.
We enabled advisory messages through:
<policyEntry topic=">" memoryLimit="5mb" producerFlowControl="true"
advisoryWhenFull="true" advisoryForSlowConsumers="true"
sendAdvisoryIfNoConsumers="true">
And sendFailIfNoSpace:
<systemUsage>
<systemUsage sendFailIfNoSpace="true">
<memoryUsage>
<memoryUsage limit="50 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="1 gb" name="foo" store="#store"/>
</storeUsage>
<tempUsage>
<tempUsage limit="100 mb"/>
</tempUsage>
</systemUsage>
</systemUsage>
We also have a monitoring process that listens to advisory messages related to
the concerned TOPIC.
After some run, we end up with OutOfMemory.
OutOfMemory analysis shows:
- 3372 instances of org.apache.activemq.command.ActiveMQTopic occupying 23,7%
Names of Topic contain ActiveMQ.Advisory.FULL.Topic:
-
ActiveMQ.Advisory.FULL.Topic.ActiveMQ.Advisory.FULL.Topic.ActiveMQ.Advisory.FULL.....ActiveMQ.Advisory.Producer.Topic.OFBTopic
1069 instances of org.apache.activemq.thread.DedicatedTaskRunner$1 holding 12%
through the names of the thread that are soooooo long.
Debuging ActiveMQ, I noticed that AMQ didn't check when sending advisory
messages that advisory concerns an advisory.
Example, suppose you set a listener for advisory message
ActiveMQ.Advisory.FULL.Topic.OFBTopic and this listener becomes slow for a
reason, then AMQ will send an advisory on this advisory provoking more and more
traffic
on the broker only for this advisory.
I attached a Stack Trace showing that an advisory will be sent for the advisory.
I will be checking to see if this issue also exists on 5.3.0.
Thanks in advance.
Philippe.
http://www.ubik-ingenierie.com
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
