[
https://issues.apache.org/jira/browse/AMQ-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173767#comment-15173767
]
Gary Tully commented on AMQ-4495:
---------------------------------
[~cshannon] yes to both.
It was oom that brought me back, then i saw the producer flow control (pfc) use
case and was convinced.
The original perf issue is a case where a consumer takes a page and as the acks
come back, the page is empty so we go to the store, take in one message (as we
again hit our limit (b/c the prefetched messages are not acked yet) and
dispatch it. Then repeat.
I think the pref issue can be addressed with some taking account of the
prefetch, but it may be tricky.
I am now thinking that we need a high and low water mark. using the low for
caching on the producer side, and using the high for page in. That is sort of
what we have with the high and full mark at the moment. so maybe there is no
need to change much.
however the full trigger is what causes pfc to kick in so it is not isolated.
Thinking now, the pfc check could be conditional on the cursor caching
messages. That may separate the concerns some which would help simplify.
Need to investigate that a bit. Thanks for your eyes on this, it is a tricky
area, dispatch on memory limits. Not sure there is a perfect answer but I think
it can be improved.
> Improve cursor memory management
> --------------------------------
>
> Key: AMQ-4495
> URL: https://issues.apache.org/jira/browse/AMQ-4495
> Project: ActiveMQ
> Issue Type: Bug
> Reporter: Dejan Bosanac
> Assignee: Gary Tully
> Fix For: 5.9.0
>
> Attachments: FasterDispatchTest.java
>
>
> As currently stands, the store queue cursor will cache producer messages
> until it gets to the 70% (high watermark) of its usage. After that caching
> stops and messages goes only in store. When consumers comes, messages get
> dispatched to it, but memory isn't released until they are acked. The problem
> is with the use case where producer flow control is off and we have a
> prefetch large enough to get all our messages from the cache. Then, basically
> the cursor gets empty and as message acks release memory one by one, we go to
> the store and try to batch one message at the time. You can guess that things
> start to be really slow at that point.
> The solution for this scenario is to wait with batching until we have more
> space so that store access is optimized. We can do this by adding a new limit
> (smaller then the high watermark) which will be used as the limit after which
> we start filling cursor from the store again.
> All this led us to the following questions:
> 1. Why do we use 70% as the limit (instead of 100%) when we stop caching
> producer messages?
> 2. Would a solution that stop caching producer messages at 100% of usage and
> then start batching messages from the store when usage drops below high
> watermark value be enough. Of course, high watermark would be configurable,
> but 100% by default so we don't alter any behavior for regular use cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
