[
https://issues.apache.org/jira/browse/KAFKA-8512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang updated KAFKA-8512:
---------------------------------
Parent: (was: KAFKA-8179)
Issue Type: Improvement (was: Sub-task)
> Looking into the Future: Assignor Version
> -----------------------------------------
>
> Key: KAFKA-8512
> URL: https://issues.apache.org/jira/browse/KAFKA-8512
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Guozhang Wang
> Priority: Major
>
> I'd propose to modify the JoinGroup protocol in this KIP as well to take the
> read `protocol version` from the PartitionAssignor.
> And then on the broker side, when choosing the leader it will pick the one
> with the highest protocol version instead of picking it "first come first
> serve".
> Although this change will not benefit the upgrade path at this time, in the
> future if we need to upgrade the assignor again, as long as they would not
> change the rebalance semantics (e.g. like we did in this KIP, from "eager" to
> "cooperative") we can actually use one rolling bounce instead since as long
> as there's one member on the newer version, that consumer will be picked.
> For example, this can also help saving "version probing" cost on Streams as
> well: suppose we augment the join group schema with `protocol version` in
> Kafka version 2.3, and then with both brokers and clients being in version
> 2.3+, on the first rolling bounce where subscription and assignment schema
> and / or user metadata has changed, this protocol version will be bumped. On
> the broker side, when receiving all member's join-group request, it will
> choose the one that has the highest protocol version (also it assumes higher
> versioned protocol is always backward compatible, i.e. the coordinator can
> recognize lower versioned protocol as well) and select it as the leader. Then
> the leader can decide, based on its received and deserialized subscription
> information, how to assign partitions and how to encode the assignment
> accordingly so that everyone can understand it. With this, in Streams for
> example, no version probing would be needed since we are guaranteed the
> leader knows everyone's version -- again it is assuming that higher versioned
> protocol is always backward compatible -- and hence can successfully do the
> assignment at that round.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
