[
https://issues.apache.org/jira/browse/KAFKA-21169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax updated KAFKA-21169:
------------------------------------
Labels: (was: group-coordinator streams)
> Streams group coordinator may trigger unnecessary rebalances on every
> heartbeat when a custom assignor returns empty task sets
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-21169
> URL: https://issues.apache.org/jira/browse/KAFKA-21169
> Project: Kafka
> Issue Type: Bug
> Components: group-coordinator, streams
> Reporter: sanghyeok An
> Assignee: sanghyeok An
> Priority: Minor
>
> Custom task assignors may use different representations for a subtopology
> with no assigned tasks. Some implementations omit the subtopology entry,
> while others include it with an empty task set:
>
> {code:java}
> // No standby tasks: omit the subtopology entry.
> Map.of()
> // No standby tasks: include the subtopology with an empty set.
> Map.of("subtopology-1", Set.of()){code}
> Both representations assign zero standby tasks. The public MemberAssignment
> API does not explicitly require empty subtopology entries to be omitted, and
> the coordinator accepts and persists them.
> However, the target assignment preserves empty entries, while reconciliation
> omits them from the current assignment. TasksTuple.sameTasks() treats these
> representations as different. As a result, an otherwise stable Streams group
> can trigger an unnecessary assignment refinement on every subsequent
> heartbeat, even though its actual task assignment has not changed.
> This advances the group, target assignment, and member epochs and generates
> additional coordinator records. Reconciliation leaves the same representation
> difference in place, so the next heartbeat triggers another update.
> Here, an unnecessary rebalance means an epoch and assignment update in the
> coordinator. In the reproduction, the assignor was called only once; the
> subsequent updates did not require rerunning it. Task restarts or data loss
> have not been demonstrated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
