[
https://issues.apache.org/jira/browse/KAFKA-10716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
A. Sophie Blee-Goldman closed KAFKA-10716.
------------------------------------------
> Streams processId is unstable across restarts resulting in task mass migration
> ------------------------------------------------------------------------------
>
> Key: KAFKA-10716
> URL: https://issues.apache.org/jira/browse/KAFKA-10716
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 2.6.0
> Reporter: A. Sophie Blee-Goldman
> Assignee: A. Sophie Blee-Goldman
> Priority: Critical
> Fix For: 2.8.0, 2.7.1, 2.6.2
>
>
> The new high availability feature of KIP-441 relies on deterministic
> assignment to produce an eventually-stable assignment. The
> HighAvailabilityTaskAssignor assigns tasks based on the unique processId
> assigned to each client, so if the same set of Kafka Streams applications
> participate in a rebalance it should generate the same task assignment every
> time.
> Unfortunately the processIds aren't stable across restarts. We generate a
> random UUID in the KafkaStreams constructor, so each time the process starts
> up it would be assigned a completely different processId. Unless this new
> processId happens to be in exactly the same order as the previous one, a
> single bounce or crash/restart can result in a large scale shuffling of tasks
> based on a completely different eventual assignment.
> Ultimately we should fix this via KAFKA-10121, but that's a nontrivial
> undertaking and this bug merits some immediate relief if we don't intend to
> tackle the larger problem in the upcoming releases
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
