[
https://issues.apache.org/jira/browse/KAFKA-20719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18110905#comment-18110905
]
Ken Huang commented on KAFKA-20719:
-----------------------------------
I'm planning to backport this to 4.3.2. This is my first time updating these
Jira fields, so I may have misunderstood the workflow.
My intention was to mark 4.3.2 in "Affects Version" to indicate that I'd like
to backport the fix there. Since the backport hasn't been completed yet, I
haven't set "Fix Version". If that's not the correct way to handle it, please
let me know.
> "Streams" group sticky assignor not sticky at startup
> -----------------------------------------------------
>
> Key: KAFKA-20719
> URL: https://issues.apache.org/jira/browse/KAFKA-20719
> Project: Kafka
> Issue Type: Bug
> Components: group-coordinator, streams
> Affects Versions: 4.2.0, 4.3.0
> Reporter: Matthias J. Sax
> Assignee: Ziyun Fu
> Priority: Critical
> Fix For: 4.4.0
>
>
> For "streams" groups (KIP-1071), the sticky-assignor is not sticky at startup
> of a new KS instance/client.
> In the "classic" protocol, a newly starting client encodes what task
> directories it finds in its local state directory as part of the rebalance
> metadata. This allows the "classic" client-side sticky assignor to re-assign
> tasks to previous clients, meeting the goal of being sticky.
> In contrast, for "streams" group no such information is encoded. A starting
> instance correctly sends empty previous assignment on its initial "join group
> heartbeat" (with epoch=0). While the group coordinator does pass in the
> "current/previous assignment" into the sticky assignor, new clients/threads
> get a new memberId, and thus previous state cannot be re-assigned in a sticky
> manner on startup. (Note, static membership might address this, but it's also
> not yet supported for "streams" group:
> https://issues.apache.org/jira/browse/KAFKA-20169)
> To fix this issue, we should encode offset information(derived from the local
> task offset checkpoint) via `taskOffsetSum` field in the "join group
> heartbeat", and update the sticky-assignor to use this data.
> This ticket is blocked by https://issues.apache.org/jira/browse/KAFKA-20116
> which does add support for `taskOffsetSum` client and broker side already, to
> support warmup tasks. After KAFKA-20116 was completed, this ticket would need
> to fix the sticky-assignor itself to use the newly available data.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
