[
https://issues.apache.org/jira/browse/AMQ-6062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dumitru HUSLEAG updated AMQ-6062:
---------------------------------
Description:
Hello,
The Broker may enter a state of high CPU consumption (100% or 200% if 2 browse
enter the infinite loop in the same time on a multi-core) when a command line
brose on multiple ques is issued. When that arives the only way we found to
restore the situation was to kill the broker and restart because he was not
responding anymore to clients. Killing the clients and the browse process does
not lower the CPU usage of the broker.
I assume the infinite loop based on successive thread dumps that show the
browse thread remaines in the same code zone:
Another IBM Link:
http://www-01.ibm.com/support/docview.wss?uid=swg21597581
{quote}
at java/util/HashMap.rehash(HashMap.java:782(Compiled Code))
at java/util/HashMap.rehash(HashMap.java:819(Compiled Code))
at java/util/HashMap.putImpl(HashMap.java:702(Compiled Code))
at java/util/HashMap.put(HashMap.java:680(Compiled Code))
*at
org/apache/activemq/broker/region/QueueBrowserSubscription.isDuplicate(QueueBrowserSubscription.java:72(Compiled
Code))
*at org/apache/activemq/broker/region/Queue.iterate(Queue.java:1688(Compiled
Code))*
at
org/apache/activemq/thread/PooledTaskRunner.runTask(PooledTaskRunner.java:133(Compiled
Code)).
{quote}
Tested versions: 5.9.1 and the last, 5.12.0, but of course I suppose all in
beetwen are affected.
The cause seems to be the non-threadsafe usage of the audit HashMap member of
QueueBrowserSubscription.java class.
The scenario may be produced with a fresh non modified install (default config)
of ActiveMQ broker:
1. set ACTIVEMQ_HOME, JAVA_HOME and PATH to point accordingly to the right
activemq, and Java version.
2. cd $ACTIVEMQ_HOME/bin; ./activemq start
3. inject at least 2000 messages in each of two ques, say QA.ONE and QA.TWO
4. $ $ACTIVEMQ_HOME/bin/activemq browse --view JMSDestination,JMSMessageID
--amqurl tcp://localhost:61616 QA.*
was:
Hello,
The Broker may enter a state of high CPU consumption (100% or 200% if 2 browse
enter the infinite loop in the same time on a multi-core) when a command line
brose on multiple ques is issued. When that arives the only way we found to
restore the situation was to kill the broker and restart because he was not
responding anymore to clients. Killing the clients and the browse process does
not lower the CPU usage of the broker.
I assume the infinite loop based on successive thread dumps that show the
browse thread remaines in the same code zone:
Another IBM Link:
http://www-01.ibm.com/support/docview.wss?uid=swg21597581
{quote}
at java/util/HashMap.rehash(HashMap.java:782(Compiled Code))
at java/util/HashMap.rehash(HashMap.java:819(Compiled Code))
at java/util/HashMap.putImpl(HashMap.java:702(Compiled Code))
at java/util/HashMap.put(HashMap.java:680(Compiled Code))
at
org/apache/activemq/broker/region/*QueueBrowserSubscription.isDuplicate(QueueBrowserSubscription.java:72(Compiled
Code))*
at org/apache/activemq/broker/region/Queue.iterate(Queue.java:1688(Compiled
Code))
at
org/apache/activemq/thread/PooledTaskRunner.runTask(PooledTaskRunner.java:133(Compiled
Code)).
{quote}
Tested versions: 5.9.1 and the last, 5.12.0, but of course I suppose all in
beetwen are affected.
The cause seems to be the non-threadsafe usage of the audit HashMap member of
QueueBrowserSubscription.java class.
The scenario may be produced with a fresh non modified install (default config)
of ActiveMQ broker:
1. set ACTIVEMQ_HOME, JAVA_HOME and PATH to point accordingly to the right
activemq, and Java version.
2. cd $ACTIVEMQ_HOME/bin; ./activemq start
3. inject at least 2000 messages in each of two ques, say QA.ONE and QA.TWO
4. $ $ACTIVEMQ_HOME/bin/activemq browse --view JMSDestination,JMSMessageID
--amqurl tcp://localhost:61616 QA.*
> Broker goes 100% CPU on multi-queue command line browse
> -------------------------------------------------------
>
> Key: AMQ-6062
> URL: https://issues.apache.org/jira/browse/AMQ-6062
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.9.1, 5.12.0
> Environment: Linux Ubuntu, Aix, with both Java 6 and Java 7 (either
> IBM or Oracle)
> Reporter: Dumitru HUSLEAG
> Priority: Critical
> Attachments: ActiveMQ_BrowseInfiniteLoopUseCase.jmx,
> ThreadStatusAnalysis_2015-11-25_14h13m33s.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Hello,
> The Broker may enter a state of high CPU consumption (100% or 200% if 2
> browse enter the infinite loop in the same time on a multi-core) when a
> command line brose on multiple ques is issued. When that arives the only way
> we found to restore the situation was to kill the broker and restart because
> he was not responding anymore to clients. Killing the clients and the browse
> process does not lower the CPU usage of the broker.
> I assume the infinite loop based on successive thread dumps that show the
> browse thread remaines in the same code zone:
> Another IBM Link:
> http://www-01.ibm.com/support/docview.wss?uid=swg21597581
> {quote}
> at java/util/HashMap.rehash(HashMap.java:782(Compiled Code))
> at java/util/HashMap.rehash(HashMap.java:819(Compiled Code))
> at java/util/HashMap.putImpl(HashMap.java:702(Compiled Code))
> at java/util/HashMap.put(HashMap.java:680(Compiled Code))
> *at
> org/apache/activemq/broker/region/QueueBrowserSubscription.isDuplicate(QueueBrowserSubscription.java:72(Compiled
> Code))
> *at org/apache/activemq/broker/region/Queue.iterate(Queue.java:1688(Compiled
> Code))*
> at
> org/apache/activemq/thread/PooledTaskRunner.runTask(PooledTaskRunner.java:133(Compiled
> Code)).
> {quote}
> Tested versions: 5.9.1 and the last, 5.12.0, but of course I suppose all in
> beetwen are affected.
> The cause seems to be the non-threadsafe usage of the audit HashMap member of
> QueueBrowserSubscription.java class.
> The scenario may be produced with a fresh non modified install (default
> config) of ActiveMQ broker:
> 1. set ACTIVEMQ_HOME, JAVA_HOME and PATH to point accordingly to the right
> activemq, and Java version.
> 2. cd $ACTIVEMQ_HOME/bin; ./activemq start
> 3. inject at least 2000 messages in each of two ques, say QA.ONE and QA.TWO
> 4. $ $ACTIVEMQ_HOME/bin/activemq browse --view JMSDestination,JMSMessageID
> --amqurl tcp://localhost:61616 QA.*
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
