[
https://issues.apache.org/jira/browse/KAFKA-21255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lucas Brutschy resolved KAFKA-21255.
------------------------------------
Resolution: Fixed
> Streams heartbeat retry responses can omit assignments after response loss
> --------------------------------------------------------------------------
>
> Key: KAFKA-21255
> URL: https://issues.apache.org/jira/browse/KAFKA-21255
> Project: Kafka
> Issue Type: Bug
> Reporter: sanghyeok An
> Assignee: sanghyeok An
> Priority: Major
> Labels: group-coordinator, streams
>
> When a Streams group coordinator applies an assignment change but the
> heartbeat response is lost, the server and client can retain different
> assignments.
> After a heartbeat failure, the client retries with its current active,
> standby, and warmup tasks. However, the server has already applied the
> assignment change while processing the previous request. If processing the
> retry produces no further assignment change, the server omits the task fields
> from its response. Consequently, the assignment that the client missed is not
> delivered again.
> The following scenarios illustrate the issue. These examples use only active
> tasks; the client reports empty lists ({{{}[]{}}}) for standby and warmup
> tasks when retrying.
>
> *Scenario 1: Losing an assignment response that advances the member epoch*
> The client owns task {{[0]}} at member epoch {{{}1{}}}. While processing a
> heartbeat, the server advances the member epoch to {{2}} and assigns tasks
> {{{}[0, 1]{}}}, but the response is lost.
> The client retries with its existing epoch {{1}} and owned task {{{}[0]{}}}.
> Since the server has already recorded epoch {{2}} and assignment {{{}[0,
> 1]{}}}, the retry response returns epoch {{2}} but omits the task fields.
> The client updates its epoch to {{2}} while continuing to own only task
> {{{}[0]{}}}. The server considers task {{1}} assigned, but the client does
> not process it. This discrepancy can persist even while subsequent heartbeats
> succeed.
>
> *Scenario 2: Losing a revocation response within the same member epoch*
> The client owns tasks {{[0, 1]}} at member epoch {{{}1{}}}, and its new
> target assignment is {{{}[0]{}}}. The server transitions the member to
> {{{}UNREVOKED_TASKS{}}}, waiting for task {{1}} to be released, and responds
> with epoch {{1}} and the remaining task {{{}[0]{}}}. This response is lost.
> Because the client did not receive the revocation instruction, it retries
> with epoch {{1}} and owned tasks {{{}[0, 1]{}}}. The server remains in the
> same state and omits the task fields because its assignment has not changed.
> The client continues to own task {{1}} and cannot acknowledge its revocation.
> Moving that task to another member is delayed. When the existing rebalance
> timeout expires, the member is removed and must rejoin.
>
> *Scenario 3: Losing an additional assignment within the same member epoch*
> The client has already reached member epoch {{{}2{}}}, but the member is in
> {{{}UNRELEASED_TASKS{}}}, waiting for another member to release a task. Once
> the previous owner releases it, the server assigns the task without changing
> member epoch {{{}2{}}}. This response is lost.
> The client retries with epoch {{2}} and the tasks it owned before the
> additional assignment. Since the server has already recorded the assignment,
> it omits the task fields from the retry response.
> The server and client have the same member epoch but different assignments.
> The client does not process the task that the server considers assigned, and
> successful heartbeats alone may not resolve the discrepancy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
