[
https://issues.apache.org/jira/browse/KAFKA-20145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans updated KAFKA-20145:
-----------------------------------
Description:
We need to send acks when receiving the same target assignment from GC, to
cover the case where a member may get the same assignment after being fenced
To achieve this, there is logic in the client state machine reconciliation, to
ack even if no change/progress. That would be ok when the reconciliation was
only triggered when a HB with a new assignment was received from the GC, but
with reconciliations triggered from the poll loop this condition may generate
noisy acks (harmless but unneeded).
At the moment, the consumer that gets a topicID-1 to reconcile and cannot
resolve metadata, will alternate between reconciling-ack-reconciling
continuously (acking back a same assignment, but triggered from poll, not
because the GC sends a new one)
Should be a minor fix and test
This is the logic that generates the ack on reconciliation attempts
[https://github.com/apache/kafka/blob/1515fddef07598ea56508fb82d2fb3546a52f831/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractMembershipManager.java#L845-L852]
(needed if the reconciliation attempt is triggered from a HB, but not needed
if triggered from poll I imagine)
was:
We need to send acks when receiving the same target assignment from GC, to
cover the case where a member may get the same assignment after being fenced
To achieve this, there is logic in the client state machine reconciliation, to
ack even if no change/progress. That used to be ok when the reconciliation was
only triggered when a HB with a new assignment was received from the GC, but
the moment we changed to trigger reconciliations from the poll loop this
condition may generate noisy acks (harmless but unneeded).
At the moment, the consumer that gets a topicID-1 to reconcile and cannot
resolve metadata, will alternate between reconciling-ack-reconciling
continuously (acking back a same assignment, but triggered from poll, not
because the GC sends a new one)
Should be a minor fix and test
> Improve consumer HB ack on unchanged target assignment
> ------------------------------------------------------
>
> Key: KAFKA-20145
> URL: https://issues.apache.org/jira/browse/KAFKA-20145
> Project: Kafka
> Issue Type: Task
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Lianet Magrans
> Priority: Minor
>
> We need to send acks when receiving the same target assignment from GC, to
> cover the case where a member may get the same assignment after being fenced
> To achieve this, there is logic in the client state machine reconciliation,
> to ack even if no change/progress. That would be ok when the reconciliation
> was only triggered when a HB with a new assignment was received from the GC,
> but with reconciliations triggered from the poll loop this condition may
> generate noisy acks (harmless but unneeded).
> At the moment, the consumer that gets a topicID-1 to reconcile and cannot
> resolve metadata, will alternate between reconciling-ack-reconciling
> continuously (acking back a same assignment, but triggered from poll, not
> because the GC sends a new one)
> Should be a minor fix and test
> This is the logic that generates the ack on reconciliation attempts
> [https://github.com/apache/kafka/blob/1515fddef07598ea56508fb82d2fb3546a52f831/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractMembershipManager.java#L845-L852]
> (needed if the reconciliation attempt is triggered from a HB, but not needed
> if triggered from poll I imagine)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
