[
https://issues.apache.org/jira/browse/KAFKA-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937299#comment-16937299
]
ASF GitHub Bot commented on KAFKA-8179:
---------------------------------------
ableegoldman commented on pull request #7386: KAFKA-8179: Part 7, cooperative
rebalancing in Streams
URL: https://github.com/apache/kafka/pull/7386
Some key points:
- allow PARTITIONS_REVOKED to transition to itself (callback is no-op if
revokedPartitions is empty)
-tasks will remain available for IQ during a rebalance (but not during
restore)
- in COOPERATIVE, we don't directly encode active tasks at all and instead
infer them from the Subscription's `ownedPartitions` and don't reassign any
partitions that were previously owned during a rebalance until they have safely
been revoked
-modify ConsumerPartitionAssignor interface to allow it to access (and
modify) the ownedPartitions and revokedPartitions (Streams can use to remove
any paused partitions from the subscription/revoked, as they can safely be
given away without first being revoked -- this is currently OFF pending further
testing/validation)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Incremental Rebalance Protocol for Kafka Consumer
> -------------------------------------------------
>
> Key: KAFKA-8179
> URL: https://issues.apache.org/jira/browse/KAFKA-8179
> Project: Kafka
> Issue Type: Improvement
> Components: consumer
> Reporter: Guozhang Wang
> Assignee: Guozhang Wang
> Priority: Major
>
> Recently Kafka community is promoting cooperative rebalancing to mitigate the
> pain points in the stop-the-world rebalancing protocol. This ticket is
> created to initiate that idea at the Kafka consumer client, which will be
> beneficial for heavy-stateful consumers such as Kafka Streams applications.
> In short, the scope of this ticket includes reducing unnecessary rebalance
> latency due to heavy partition migration: i.e. partitions being revoked and
> re-assigned. This would make the built-in consumer assignors (range,
> round-robin etc) to be aware of previously assigned partitions and be sticky
> in best-effort.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
