[ https://issues.apache.org/jira/browse/AMQ-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Tully updated AMQ-4495: ---------------------------- Comment: was deleted (was: [~cshannon] {quote} I've seen out of memory problems occasionally even though proper usage limits are set {quote} This bug could also be related to what you observed [ENTMQ-1526|https://issues.jboss.org/browse/ENTMQ-1526].) > Improve cursor memory management > -------------------------------- > > Key: AMQ-4495 > URL: https://issues.apache.org/jira/browse/AMQ-4495 > Project: ActiveMQ > Issue Type: Bug > Reporter: Dejan Bosanac > Assignee: Gary Tully > Priority: Major > Fix For: 5.9.0 > > Attachments: FasterDispatchTest.java > > > As currently stands, the store queue cursor will cache producer messages > until it gets to the 70% (high watermark) of its usage. After that caching > stops and messages goes only in store. When consumers comes, messages get > dispatched to it, but memory isn't released until they are acked. The problem > is with the use case where producer flow control is off and we have a > prefetch large enough to get all our messages from the cache. Then, basically > the cursor gets empty and as message acks release memory one by one, we go to > the store and try to batch one message at the time. You can guess that things > start to be really slow at that point. > The solution for this scenario is to wait with batching until we have more > space so that store access is optimized. We can do this by adding a new limit > (smaller then the high watermark) which will be used as the limit after which > we start filling cursor from the store again. > All this led us to the following questions: > 1. Why do we use 70% as the limit (instead of 100%) when we stop caching > producer messages? > 2. Would a solution that stop caching producer messages at 100% of usage and > then start batching messages from the store when usage drops below high > watermark value be enough. Of course, high watermark would be configurable, > but 100% by default so we don't alter any behavior for regular use cases. -- This message was sent by Atlassian Jira (v8.3.2#803003)