We are getting similar problems in ActiveMQ 3.2.1. We have a similar scenario to Frank, where our consumers may be down (e.g. for maintenance) but our producers will continue to produce messages. We also want the message broker to be a buffer to absorb surges in load, until such time that our consumers can catch up (e.g. during quiet times overnight). We are sizing our message consuming for the average daily load, not the peak.
In the 3.2.1 release, the maxMem attribute on the broker definition seems to control how many messages we can get into the queue. I am reasonably sure that persistence is working, as I can see the journal files being updated and I can see the messages in the ACTIVEMQ_MSGS table in the database (Oracle). I have 10 x 40 MB log files and plenty of capacity in Oracle. My messages are quite small (1 - 2 KB) and are text messages. I have my maxMem set to the equivalent of 80 MB (in bytes). I was expecting that with the amount of journal capacity and the Oracle database that I would be able to get in the millions of entries into the queue when there was no consumers, and that they would simply "queue up". What actually happens is that around 60,000 messages I start getting lots of messages in the log: 2006-06-02 16:21:22,369 [Thread-0 ] WARN ientQueueBoundedMessageManager - memory limit low - forced to remove expired messages: publish.request Around about 65,000 messages I start getting lots of: 2006-06-02 17:19:05,159 [ocalport=61616]] WARN MemoryBoundedQueue - Queue is full, waiting for it to be dequeued. At this point, my producer seems to lock up and everything stops. When I restart ActiveMQ, my messages don't seem to come back. I can see the 65,000 messages in the ACTIVEMQ_MSGS table in Oracle, but my consumer can't seem to dequeue them. This seems to be covered in http://issues.apache.org/activemq/browse/AMQ-402 but won't be fixed in the release 3 codeline. I have tried restarting several times and removing the journal files, but nothing gets them back. Is there any way to configure ActiveMQ (3.2.x) to reliably queue up a number of messages, only limited by disk/database capacity? The application that we are developing could produce average loads of 30 messages/second, so 65,000 messages is nothing. Even if we up the maxMem, we will still have a limit - and the bigger the maxMem, the worse the consequences are if we reach it as we will lose more messages. Upgrading to 4 is not an option for us at the moment, as we are in final stages of testing and don't want to have to re-test with a version that is not final release. -- View this message in context: http://www.nabble.com/Slow-consumer-problem-t1733329.html#a4724627 Sent from the ActiveMQ - User forum at Nabble.com.
