On Tue, Mar 7, 2017 at 5:27 PM, ipolevoy <i...@expresspigeon.com> wrote:
> In our projects, we use two queues, one for normal processing, and another > for errors. The "error" queue sometimes gets filled with "error" messages, > which are sitting there, until a human checks them out. In some cases, the > "error" queue fills out with a lot of messages, and then the JVM runs out > of > heap. > For instance, currently, the JVMs configured with max 4GB heap, and get OOM > exception at random times. We used a MemoryAnalyzer as well as IBM Heap > Analyzer and both point to Artemis. When I checked the size of the journal > on the file system, it is about 5GB. > > We send: > > * Small messages > * Persistent > * Not using Netty or remoting (just use Artemis for async processing) > * So, my questions are about Apache Artemis heap management and > recommendations: > > 1. Does it store persistent messages in RAM as well, even if they are > already stored on the file system? > There are 3 reasons Artemis persists messages. 1. In the Message journal, the message journal is used only to recover application state if the server is shutdown. 2. Large Messages can be streamed directly to disk to avoid taking up lots of memory 3. Paging when an address runs out of memory Artemis (if appropriately configured) will stop storing messages in memory and instead start writing them directly to disk. N.B. If messages are written directly to disk when Artemis receives, and *not* stored in memory, we'd need to keep reading them from disk when delivering to consumers. We have some optimisations when doing paging by using caches, but it is still much slower. 2. If #1 is true, what is the strategy to control/limit amount od RAM > allocated by Artemis? > There are multiple strategies (address-full-policies) for protecting the broker from OOM issues. Check out the address-full-policy attribute of address-settings: https://activemq.apache.org/artemis/docs/1.5.3/queue-attributes.html >From what you've written here, I'd suggest enabling PAGING on your error queue: https://activemq.apache.org/artemis/docs/1.5.3/paging.html. This will allow the queue to grow large, until a user batch processes the errors. Also, this message is posted on SO: > http://stackoverflow.com/questions/42497606/how-does- > apache-artemis-manage-heap-space-ram > > Appreciate any help! > > Regards Martyn > > > -- > View this message in context: http://activemq.2283324.n4. > nabble.com/How-does-Apache-Artemis-manage-heap-space-RAM-tp4723220.html > Sent from the ActiveMQ - User mailing list archive at Nabble.com. >
