I am using 2 ActiveMQ brokers 5.5.0 with failover on Windows. I have a C# windows service which connects to the broker queue via NMS. Windows service creates 100 threads and each thread has its own consumer. I use AutoAcknowledge mode. I run 4 instances of the windows service on separate machines. PrefetchSize is set to 1 and I use synchronous receive method with small (10 ms) wait time.
I also have the following setup on brokers for queues: <policyEntry queue=">" producerFlowControl="false" memoryLimit="1mb"> After working for some period of time (usually a day or two) all consumers stop receiving any messages, but I can see messages in brokers. Usually all windows service instances stop receiving messages simultaneously, but not every time. If I restart services then they start processing messages. I've created a dump file and I found a lot of threads are waiting for a mutex to be released. Here is the stack trace: System.Threading.Monitor.Wait(System.Object, System.TimeSpan) Apache.NMS.ActiveMQ.Util.SimplePriorityMessageDispatchChannel.Dequeue(System.TimeSpan) Apache.NMS.ActiveMQ.MessageConsumer.Dequeue(System.TimeSpan) Apache.NMS.ActiveMQ.MessageConsumer.Receive(System.TimeSpan) I also have a lot of DedicatedTaskRunners waiting for something: System.Threading.Monitor.Wait(System.Object) Apache.NMS.ActiveMQ.Threads.DedicatedTaskRunner.Run() System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean) System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) System.Threading.ThreadHelper.ThreadStart() I've found a few issues with the same problem, but I can't find my case: 1) http://activemq.2283324.n4.nabble.com/Client-threads-hang-if-ActiveMQ-dies-td2363080.html - I don't use transacted messages anymore, neither prefetchSize=0. I don't use ReceiveNoWait as well (though I could move waiting logic to my application), because it seems to cause some problems. 2) http://activemq.2283324.n4.nabble.com/Consumer-Receive-hangs-td2364683.html - I set producerFlowControl to false, and afair the broker could use as much memory as the server has. I also found an issue which describes the same problem and is marked as resolved - https://issues.apache.org/jira/browse/AMQ-2446 What else could cause such a problem? -- View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-NMS-consumer-hangs-after-sometime-tp4364689p4364689.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
