[
https://issues.apache.org/jira/browse/ARTEMIS-4314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733204#comment-17733204
]
Gary Tully commented on ARTEMIS-4314:
-------------------------------------
[~tabish] had a great observation to suggest that with local producers keeping
local consumers busy, messages on the upstream will be stranded b/c the local
queue will never be empty.
That is fair, and I think a solution is to replace the check for empty with a
check for pendingMessageCount == 0, such that if we have spare capacity, we
pull a batch. If local producers are keeping us full, we will still leave the
messages stranded, but in that case we are already at capacity. I think that is
a better match.
thoughts?
> Federation, support consumerWindowSize zero and federate in batches only when
> the local queue is empty
> ------------------------------------------------------------------------------------------------------
>
> Key: ARTEMIS-4314
> URL: https://issues.apache.org/jira/browse/ARTEMIS-4314
> Project: ActiveMQ Artemis
> Issue Type: Improvement
> Components: Federation
> Affects Versions: 2.28.0
> Reporter: Gary Tully
> Assignee: Gary Tully
> Priority: Major
> Fix For: 2.29.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Dual queue federation, where clusters federate in both direction can suffer
> from message flip flopping once the priority adjustment kicks in.
> If there is a large backlog, the lower priority federation consumer is in
> play once all of the local consumer credit is exhausted and the backlog can
> drain to the other cluster.
> If demand is low there, the process can repeat. limiting the rate of the
> federation consumer can help but it is not ideal b/c when there is no local
> demand, we want to have a high rate of migration.
>
> A possible solution is to have the federation consumer manage its own credit
> and only flow messages when the local queue is empty. Then flow a batch of
> messages, and await again that the local queue is empty. In this way, there
> is no thundering heard effect, but there is also fast migration of messages
> once there is demand.
> the consumerWindowSize=0 is already in play for consumer.receive calls and
> there is already a defaultConsumerWindowSize for an address. These can be
> combined to realise batchFederationOnEmpty semantics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
