[jira] [Updated] (AMQ-5343) org.apache.activemq.thread.PooledTaskRunner.shutdown Waiting state
[ https://issues.apache.org/jira/browse/AMQ-5343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Smith updated AMQ-5343: Attachment: threaddump.13574.txt I _think_ we hit something similar, see attached thread dump file (threaddump.13574.txt) The background here is that we have a process that is a consuming client of ActiveMQ 5.10.0. This process regularly shuts itself down and starts up (cleaning up some leaky resources from another area unrelated). This application which processes several thousand items and then shutsdown, to be started again by an external process has worked really well for weeks in production handling thousands of messages, but today one of these consumers hung on the attempt to shutdown cleanly. Given this shutdown happens in an orderly fashion, the message is processed, a certain count of messages processed is checked and if exceeded goes through the shutdown process, I can't see why this wouldn't just work, but perhaps there's a timing issue in there somewhere. org.apache.activemq.thread.PooledTaskRunner.shutdown Waiting state -- Key: AMQ-5343 URL: https://issues.apache.org/jira/browse/AMQ-5343 Project: ActiveMQ Issue Type: Bug Components: JMS client Affects Versions: 5.10.0 Reporter: Mohit Anchlia Attachments: threaddump.13574.txt I am using a framework with async request/response pattern where I cache session and connections. I also cache temp queue destination and use it with future to get the response. The problem seems to be when I try to close the connection I see threads are waiting forever: pool-2-thread-9 prio=10 tid=0x7f76843f4800 nid=0xadb in Object.wait() [0x7f765194d000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.activemq.thread.PooledTaskRunner.shutdown(PooledTaskRunner.java:104) - locked 0x950ddca8 (a org.apache.activemq.thread.PooledTaskRunner$1) at org.apache.activemq.thread.PooledTaskRunner.shutdown(PooledTaskRunner.java:112) at org.apache.activemq.ActiveMQSessionExecutor.stop(ActiveMQSessionExecutor.java:152) - locked 0x94f1ae10 (a org.apache.activemq.ActiveMQSessionExecutor) at org.apache.activemq.ActiveMQSession.dispose(ActiveMQSession.java:720) - locked 0x950c89f8 (a org.apache.activemq.ActiveMQSession) at org.apache.activemq.ActiveMQSession.doClose(ActiveMQSession.java:656) at org.apache.activemq.ActiveMQSession.close(ActiveMQSession.java:649) at com.sony.am.snei.stress.async.jms.JMSClient.cleanUp(JMSClient.java:172) at com.sony.am.snei.stress.command.FDBIdentityAndAccessCommand.doWrite(FDBIdentityAndAccessCommand.java:142) at com.sony.am.snei.stress.command.FDBIdentityAndAccessCommand.call(FDBIdentityAndAccessCommand.java:46) at com.sony.am.snei.stress.command.FDBIdentityAndAccessCommand.call(FDBIdentityAndAccessCommand.java:22) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMQ-5544) Broker level TotalMessageCount metric can be negative
Paul Smith created AMQ-5544: --- Summary: Broker level TotalMessageCount metric can be negative Key: AMQ-5544 URL: https://issues.apache.org/jira/browse/AMQ-5544 Project: ActiveMQ Issue Type: Bug Components: Broker Affects Versions: 5.10.0 Reporter: Paul Smith Priority: Minor We export both a broker level TotalMessageCount and per-queue QueueSize metrics to a separate Metric collection system. We recently had an issue which required us to purge the queue, and this seemed to reset the Queue level metrics well, but now the TotalMessageCount on the Broker JMX bean is now reporting a count of -24415. This makes alarming over broker level queues a bit difficult, and in fact it's now a bit useless as a metric until we can restart the broker. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMQ-5529) Deadlock using JMX copyMatchingMessagesTo
Paul Smith created AMQ-5529: --- Summary: Deadlock using JMX copyMatchingMessagesTo Key: AMQ-5529 URL: https://issues.apache.org/jira/browse/AMQ-5529 Project: ActiveMQ Issue Type: Bug Components: JMX Affects Versions: 5.10.0 Environment: Linux app1.syd.acx 2.6.39-200.24.1.el6uek.x86_64 #1 SMP Sat Jun 23 02:39:07 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux java version 1.7.0_60 Java(TM) SE Runtime Environment (build 1.7.0_60-b19) Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode) Reporter: Paul Smith Today we had an issue relating to a consumer that was somehow not properly ack'ing messages and letting the queue grow. We had the idea to use JMX operations to move messages we knew had been delivered and processed, but before hand tried to copy a few of the messages to a new destination to confirm using a selector of JMSTimestamp[timestampAsLongValueOfYesterdayMorning] (obviously a long-based timestamp I've since forgotten to write down. I tested the Selector with a browse(String selector) operation, and that successfully returned the small handful easily. I invoked the copyMatchingMessagesTo Operation on the Queue JMX operation, and the attached Thread Deadlock occurred. At the time there were around 20,000 messages in the queue, all very small in size.(roughly 1k each as reported by the JMX averageMessageSize property) This doesn't give us much confidence in the ability to use Operational JMX to recover from strange scenarios. The total number of messages being copied was a small handful, 5-20 at most. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AMQ-5529) Deadlock using JMX copyMatchingMessagesTo
[ https://issues.apache.org/jira/browse/AMQ-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Smith updated AMQ-5529: Attachment: amq_jstack.txt Deadlock using JMX copyMatchingMessagesTo --- Key: AMQ-5529 URL: https://issues.apache.org/jira/browse/AMQ-5529 Project: ActiveMQ Issue Type: Bug Components: JMX Affects Versions: 5.10.0 Environment: Linux app1.syd.acx 2.6.39-200.24.1.el6uek.x86_64 #1 SMP Sat Jun 23 02:39:07 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux java version 1.7.0_60 Java(TM) SE Runtime Environment (build 1.7.0_60-b19) Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode) Reporter: Paul Smith Attachments: amq_jstack.txt Today we had an issue relating to a consumer that was somehow not properly ack'ing messages and letting the queue grow. We had the idea to use JMX operations to move messages we knew had been delivered and processed, but before hand tried to copy a few of the messages to a new destination to confirm using a selector of JMSTimestamp[timestampAsLongValueOfYesterdayMorning] (obviously a long-based timestamp I've since forgotten to write down. I tested the Selector with a browse(String selector) operation, and that successfully returned the small handful easily. I invoked the copyMatchingMessagesTo Operation on the Queue JMX operation, and the attached Thread Deadlock occurred. At the time there were around 20,000 messages in the queue, all very small in size.(roughly 1k each as reported by the JMX averageMessageSize property) This doesn't give us much confidence in the ability to use Operational JMX to recover from strange scenarios. The total number of messages being copied was a small handful, 5-20 at most. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AMQ-5478) REST endpoint creates topic instead of queue
Paul Smith created AMQ-5478:
---
Summary: REST endpoint creates topic instead of queue
Key: AMQ-5478
URL: https://issues.apache.org/jira/browse/AMQ-5478
Project: ActiveMQ
Issue Type: Bug
Components: webconsole
Affects Versions: 5.10.0
Reporter: Paul Smith
Priority: Minor
This is _possibly_ simply a Documentation bug, but I was testing the REST
endpoints via the documentation here:
http://activemq.apache.org/rest.html
When I first tried to post a simple message to a queue (and this queue didn't
exist yet) I used this script:
{noformat}
$ curl -d body=message -XPOST
http://admin:admin@localhost:8161/api/message/queue/orders/input
{noformat}
The above is mostly based on the documentation. I was surprised to find no
queue created by looking at the web console, but i _*did*_ see a Topic created
called 'queue.orders.input' and it contained 1 message (odd because there's no
consumers defined).
After deleting the topic I then tried the 'other format' of the URL:
{noformat}
curl -d body=message -XPOST
http://admin:admin@localhost:8161/api/message/orders.input?type=queue
{noformat}
This does create a Queue of name 'orders.input' as one would expect.
I'm not exactly clear then from the documentation where in the URL one is
defining it as a queue using the former syntax. The URL in the documentation
sort of _implies_ that the 'queue' prefix of the URL is creating a queue named
'orders.input' but it doesn't turn out that way.
So either this is a bug, or the documentation needs tweaking.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
