[
https://issues.apache.org/jira/browse/KAFKA-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272901#comment-17272901
]
feyman edited comment on KAFKA-9689 at 1/28/21, 3:54 AM:
---------------------------------------------------------
The version detection flow leveraging the versioning system is as described in
the section: Use case: {{group_coordinator}} feature flag in KIP-584.
The code change mainly contains 3 parts:
1) StreamThread should know if itself is leader in the consumer group, if yes,
it should periodically query the describeFeatures api to see if there are
feature metadata updates
2) There should be some place to put the feature metadata in the
MemberMetadata, either in the assignment(userData) or add a new field in the
MemberMetadata(which involves public interface change). Current implementation
leverages the assignment.
2.1 Each streamThread put the feature metadata(EOS feature version) in the
SubscriptionInfo when subscribe
2.2 Upon receiving the JoinGroupResp, the leader will know the current feature
version in the broker side, it can put the current broker side feature
version(if updated) in the assignment as suggested feature version
2.3 when the follower receive the assignment in the SyncGroupResp, it will find
the new broker side latest feature version
3) the StreamThread should dynamically switch to the new thread producer
without affecting the existing tasks that
I'm implementing the code as the sequence above, currently on 2, but need to
discuss if step 2 make sense, haven't start step 3 yet.
Questions to [~bchen225242] :
A) 2.1 Might need to add a new field in the SubscriptionInfoData to include the
client side feature metadata, it seems ok to me since SubscriptionInfoData is
the stream-specific and doesn't seem to need a KIP for it, thoughts ?
was (Author: feyman):
The version detection flow leveraging the versioning system is as described in
the section: Use case: {{group_coordinator}} feature flag in KIP-584.
The code change mainly contains 3 parts:
1) StreamThread should know if itself is leader in the consumer group, if yes,
it should periodically query the describeFeatures api to see if there are
feature metadata updates
2) There should be some place to put the feature metadata in the
MemberMetadata, either in the assignment(userData) or add a new field in the
MemberMetadata(which involves public interface change). Current implementation
levrages the assignment.
3) the StreamThread should dynamically switch to the new thread producer
without affecting the existing tasks that
> Automatic broker version detection to initialize stream client
> --------------------------------------------------------------
>
> Key: KAFKA-9689
> URL: https://issues.apache.org/jira/browse/KAFKA-9689
> Project: Kafka
> Issue Type: New Feature
> Reporter: Boyang Chen
> Assignee: feyman
> Priority: Major
>
> Eventually we shall deprecate the flag to suppress EOS thread producer
> feature, instead we take version detection approach on broker to decide which
> semantic to use.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
