Hi Chia-Ping, Thanks for the suggestion.
The “isClassic” field in ConsumerGroupDescribeResponse is a boolean value, so we can’t use it as the protocol string. I change the “PROTOCOL” column to “IS-CLASSIC” in kafka-consumer-groups.sh. It returns “true” if a member is in a classic group or it’s a classic member in a consumer group. In other cases, it returns “false”. I have a draft implementation in https://github.com/apache/kafka/pull/17872. Thanks, PoAn > On Nov 23, 2024, at 1:05 PM, Chia-Ping Tsai <chia7...@gmail.com> wrote: > > hi PoAn > > `isClassic` is good to me :) > > Best, > Chia-Ping > > PoAn Yang <yangp...@gmail.com> 於 2024年11月23日 週六 下午12:35寫道： > >> Hi Chia-Ping / David, >> >> Thanks for the great suggestion. >> >> How about using “isClassic”? >> The term like “isLegacy” may be equivocal in the future. >> For example, if we have a newer version of consumer / share consumer, the >> meaning will be different for different cases. >> If we use “isClassic”, for MemberDescription in ClassicGroupDescription, >> the value can be always “true”. >> In ConsumerGroupDescription, it’s “true” when it’s a classic member. >> In ShareGroupDescription, it’s always “false”. >> >> Thanks, >> PoAn >> >>> On Nov 23, 2024, at 1:02 AM, Chia-Ping Tsai <chia7...@gmail.com> wrote: >>> >>> hi DJ >>> >>>> My goal was to have a way to tell the user that his (new) consumer group >>> may have non-upgraded members yet. I wonder if we could use a boolean to >>> signal this, e.g. isLegacy. Then, in the CLI we could have a column which >>> is only displayed when they are non upgraded members. What do you think? >>> >>> Yes, this is a good approach, but the downside is that the new isLegacy >>> field in MemberDescription would be irrelevant for classic and shared >>> groups. This would require thorough documentation to clarify its purpose. >>> >>> Additionally, we could use a boolean instead of a string for the new >> field >>> in the RPC to reduce the request size. >>> Best, >>> Chia-Ping >>> >>> >>> >>> David Jacot <dja...@confluent.io.invalid> 於 2024年11月23日 週六 上午12:41寫道： >>> >>>> Hi both, >>>> >>>> Ah, I understand what you mean now. You're saying that we have two >> protocol >>>> fields. One at the group level and one at the member level (the new >> one). >>>> The two fields are actually two different things. The former is the >> name of >>>> the embedded protocol used in the classic protocol whereas the latter is >>>> the name of the protocol/api used. This is indeed confusing. >>>> >>>> My goal was to have a way to tell the user that his (new) consumer group >>>> may have non-upgraded members yet. I wonder if we could use a boolean to >>>> signal this, e.g. isLegacy. Then, in the CLI we could have a column >> which >>>> is only displayed when they are non upgraded members. What do you think? >>>> >>>> If we cannot find a good way to represent this, we could drop it from >> this >>>> KIP too. >>>> >>>> Best, >>>> David >>>> >>>> On Fri, Nov 22, 2024 at 5:12 PM PoAn Yang <yangp...@gmail.com> wrote: >>>> >>>>> Hi Chia-Ping / David, >>>>> >>>>> Thanks for the review and suggestions. >>>>> >>>>>>> I don't fully follow your comment so please excuse me if I say >>>> something >>>>>>> wrong. >>>>> >>>>> I think Chia-Ping wants to say: originally, the default protocol in >>>>> classic group is “consumer”. >>>>> After we have a new consumer group, the protocol of a classic member in >>>>> consumer group is “classic”. >>>>> The MemberDescription is used by both ClassicGroupDescription and >>>>> ConsumerGroupDescription. >>>>> If we add the protocol field to MemberDescription, the protocol of a >>>>> classic member in classic group is “consumer”, but it’s “classic” in >>>>> consumer group. >>>>> To avoid misleading word, we can add a new collection of member >> protocols >>>>> to ConsumerGroupDescription only. >>>>> The original goal is to show member protocol during consumer migration, >>>> so >>>>> we don’t need to add this information to ClassicGroupDescription. >>>>> >>>>> Please correct me if I misunderstood anything. >>>>> >>>>> Thanks, >>>>> PoAn >>>>> >>>>>> On Nov 22, 2024, at 7:00 PM, Chia-Ping Tsai <chia7...@gmail.com> >>>> wrote: >>>>>> >>>>>> hi DJ >>>>>> >>>>>>> How about we use another Jira to add “SHARE” to GroupProtocol and >>>> change >>>>>> return type of ModernGroup#protocolType? >>>>>> >>>>>> This is not my comment ... sorry for the incorrect quote :( >>>>>> >>>>>>> A member in a classic group should report "classic". >>>>>> >>>>>> I completely agree. However, in this case, >>>>> ClassicGroupDescription#protocol >>>>>> returns 'consumer', while ClassicGroupDescription#members[x].protocol >>>>>> returns 'classic'. This inconsistency feels a bit strange to me. >>>>>> >>>>>> Best, >>>>>> Chia-Ping >>>>>> >>>>>> >>>>>> David Jacot <dja...@confluent.io.invalid> 於 2024年11月22日 週五 下午6:51寫道： >>>>>> >>>>>>> Hi Chia, >>>>>>> >>>>>>>>> LM4: I agree it’s better to use a type like GroupProtocol enum, but >>>> I >>>>>>> would like to align the format with kafka-groups.sh. >>>>>>> How about we use another Jira to add “SHARE” to GroupProtocol and >>>> change >>>>>>> return type of ModernGroup#protocolType? >>>>>>> >>>>>>> My understanding is that `GroupProtocol` is used to drive the config >>>> of >>>>> the >>>>>>> `Consumer`. Hence adding `Share` seems wrong to me. Or are you >>>>> referring to >>>>>>> another GroupProtocol enum? Perhaps GroupType enum? >>>>>>> >>>>>>>> In fact, I'm wondering if we should even add 'protocol' to >>>>>>> MemberDescription, >>>>>>> since the field has different values between classic and consumer >>>>> groups—it >>>>>>> shows 'consumer' in classic groups, whereas it shows 'classic' in >>>>> consumer >>>>>>> groups. Yes, it seems like a typo, but it's true. This inconsistency >>>>> feels >>>>>>> confusing to me. For example, users call Admin#describeConsumerGroups >>>>> and >>>>>>> see that a member has the 'consumer' protocol when it's in a classic >>>>> group. >>>>>>> Then, if the group is upgraded to a consumer group, calling >>>>>>> Admin#describeConsumerGroups again shows the protocol as 'classic' >> for >>>>> the >>>>>>> same member. >>>>>>> >>>>>>> I don't fully follow your comment so please excuse me if I say >>>> something >>>>>>> wrong. >>>>>>> >>>>>>> I think that it should work as follows: >>>>>>> * A member in a classic group should report "classic". >>>>>>> * A member using the consumer protocol in a consumer group should >>>> report >>>>>>> "consumer". >>>>>>> * A member using the classic protocol in a consumer group should >>>> report >>>>>>> "classic". >>>>>>> * A member in a share group should report "share". >>>>>>> >>>>>>>> If all we want to do is distinguish classic members in the >>>>>>> ConsumerGroupDescription, then we shouldn't modify the common >>>>>>> MemberDescription structure. Instead, we could add a collection to >>>>>>> ConsumerGroupDescription to trace members using the classic protocol, >>>>> which >>>>>>> would allow us to enrich the output of command-line tools. >>>>>>> >>>>>>> Personally, I believe that using different collections per protocol >>>>> types >>>>>>> is not necessary. A field in the member is more than enough. >>>>>>> >>>>>>> Best, >>>>>>> DJ >>>>>>> >>>>>>> On Fri, Nov 22, 2024 at 12:01 PM Chia-Ping Tsai <chia7...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> hi PoAn >>>>>>>> >>>>>>>>> LM4: I agree it’s better to use a type like GroupProtocol enum, but >>>> I >>>>>>>> would like to align the format with kafka-groups.sh. >>>>>>>> How about we use another Jira to add “SHARE” to GroupProtocol and >>>>> change >>>>>>>> return type of ModernGroup#protocolType? >>>>>>>> >>>>>>>> MemberDescription is also used by ClassicGroupDescription, and in >>>>> classic >>>>>>>> groups, the protocol can be an arbitrary string, so we can't use an >>>>> enum. >>>>>>>> >>>>>>>> In fact, I'm wondering if we should even add 'protocol' to >>>>>>>> MemberDescription, >>>>>>>> since the field has different values between classic and consumer >>>>>>> groups—it >>>>>>>> shows 'consumer' in classic groups, whereas it shows 'classic' in >>>>>>> consumer >>>>>>>> groups. Yes, it seems like a typo, but it's true. This inconsistency >>>>>>> feels >>>>>>>> confusing to me. For example, users call >> Admin#describeConsumerGroups >>>>> and >>>>>>>> see that a member has the 'consumer' protocol when it's in a classic >>>>>>> group. >>>>>>>> Then, if the group is upgraded to a consumer group, calling >>>>>>>> Admin#describeConsumerGroups again shows the protocol as 'classic' >>>> for >>>>>>> the >>>>>>>> same member. >>>>>>>> >>>>>>>> If all we want to do is distinguish classic members in the >>>>>>>> ConsumerGroupDescription, then we shouldn't modify the common >>>>>>>> MemberDescription structure. Instead, we could add a collection to >>>>>>>> ConsumerGroupDescription to trace members using the classic >> protocol, >>>>>>> which >>>>>>>> would allow us to enrich the output of command-line tools. >>>>>>>> Best, >>>>>>>> Chia-Ping >>>>>>>> >>>>>>>> PoAn Yang <yangp...@gmail.com> 於 2024年11月22日 週五 下午3:46寫道： >>>>>>>> >>>>>>>>> Hi Lianet / Jeff, >>>>>>>>> >>>>>>>>> Thanks for the review. >>>>>>>>> >>>>>>>>> LM5: The kafka-share-groups.sh also uses “—describe” to show >> offsets >>>>>>>>> information. >>>>>>>>> Change to use “—describe —state —verbose” to show group level >>>>>>> information >>>>>>>>> in kafka-share-groups.sh. >>>>>>>>> >>>>>>>>> LM6: Update the example. >>>>>>>>> >>>>>>>>> LM7: Add a description to mention “—verbose” is a new option in >>>>>>>>> ShareGroupCommandOptions. >>>>>>>>> >>>>>>>>> JK2: Yes, add the statement about format change for ASSIGNMENT >>>> value. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> PoAn >>>>>>>>> >>>>>>>>>> On Nov 22, 2024, at 7:35 AM, Jeff Kim <jeffkb...@apache.org> >>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi PoAn, >>>>>>>>>> >>>>>>>>>> JK2: Can we state in the KIP that we will reformat the existing >>>>>>>>>> classic group's `--members --verbose` as well, specifically the >>>>>>>>>> "ASSIGNMENT" column to include the topic name? >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> >>>>>>>>>> On 2024/11/21 14:33:37 "Lianet M." wrote: >>>>>>>>>>> Thanks for the updates PoAn! >>>>>>>>>>> >>>>>>>>>>> LM6. Just a nit, under the --describe --state --verbose the >> header >>>>>>> is >>>>>>>>> fine >>>>>>>>>>> but the example is still missing the --state argument. >>>>>>>>>>> >>>>>>>>>>> LM7. Under the Proposed changes for kafka-share-groups.sh, should >>>> we >>>>>>>>>>> clearly mention that the KIP adds a new --verbose option? I think >>>>>>> it's >>>>>>>>>>> confusing because it's presented as if the option already exists >>>> and >>>>>>>>> we're >>>>>>>>>>> only changing the output (which is the case for >>>>>>> kafka-consumer-groups >>>>>>>>> but >>>>>>>>>>> not for kafka-share-groups) >>>>>>>>>>> >>>>>>>>>>> That's all on my side. Thanks! >>>>>>>>>>> >>>>>>>>>>> Lianet >>>>>>>>>>> >>>>>>>>>>> On Wed, Nov 20, 2024 at 11:12 PM PoAn Yang <yangp...@gmail.com> >>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Lianet / Jeff, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for review and suggestions. >>>>>>>>>>>> >>>>>>>>>>>> LM4: At group level, the protocol field is from >>>>>>>>> ModernGroup#protocolType. >>>>>>>>>>>> It’s a string. >>>>>>>>>>>> The value is defined in different places like >>>>>>>> ShareGroup#PROTOCOL_TYPE >>>>>>>>> and >>>>>>>>>>>> ConsumerProtocol#PROTOCOL_TYPE. >>>>>>>>>>>> In KIP-1043 < >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=305171038#KIP1043:Administrationofgroups-kafka-groups.sh >>>>>>>>>> , >>>>>>>>>>>> the kafka-groups.sh also shows protocol like “classic” / >>>>>>> “consumer” / >>>>>>>>>>>> “share”. >>>>>>>>>>>> I agree it’s better to use a type like GroupProtocol enum, but I >>>>>>>> would >>>>>>>>>>>> like to align the format with kafka-groups.sh. >>>>>>>>>>>> How about we use another Jira to add “SHARE” to GroupProtocol >> and >>>>>>>>> change >>>>>>>>>>>> return type of ModernGroup#protocolType? >>>>>>>>>>>> >>>>>>>>>>>> LM5 / JK1: Sorry, I misunderstood LM3, so I deleted —state in >> the >>>>>>>>> title. >>>>>>>>>>>> You’re correct. I would like to use `—describe —state —verbose` >>>> to >>>>>>>> show >>>>>>>>>>>> group level information. >>>>>>>>>>>> For `—describe —verbose`, it will show output as same as >>>> `—describe >>>>>>>>>>>> —offsets —verbose`, >>>>>>>>>>>> because `—describe` also shows output as same as `—describe >>>>>>>> —offsets`. >>>>>>>>>>>> >>>>>>>>>>>> JK2: Yes, I only use consumer group in the sample output, but >> the >>>>>>>>> classic >>>>>>>>>>>> group will use the same format as well. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> PoAn >>>>>>>>>>>> >>>>>>>>>>>>> On Nov 21, 2024, at 8:28 AM, Jeff Kim <jeffkb...@apache.org> >>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi PoAn, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for the KIP. I have some questions: >>>>>>>>>>>>> >>>>>>>>>>>>> JK1: (Realized this is similar to LM5) On `--describe >> --verbose` >>>>>>>>>>>> proposed changes - doesn't `--describe $group` default to >>>> printing >>>>>>>> the >>>>>>>>>>>> offsets? Perhaps you're referring to the `--state` argument? >>>> Also, >>>>>>>>> would >>>>>>>>>>>> that mean the default `--describe $group --verbose` command >> would >>>>>>> not >>>>>>>>> print >>>>>>>>>>>> the added field to `--offsets verbose` (leader-epoch) or would >>>> it? >>>>>>>>>>>>> >>>>>>>>>>>>> JK2: Looking at ConsumerGroupCommand.java, the existing >>>>>>> "ASSIGNMENT" >>>>>>>>>>>> column under `--members --verbose` does group the topic >>>> partitions >>>>>>> by >>>>>>>>> topic >>>>>>>>>>>> but does not prefix the grouping with the topic name like you're >>>>>>>>> proposal: >>>>>>>>>>>> "my_topic:0,1;new_topic:0,1". Should we do apply the same format >>>>>>> for >>>>>>>>> the >>>>>>>>>>>> classic group as well? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2024/11/20 21:32:41 "Lianet M." wrote: >>>>>>>>>>>>>> Hello PoAn, just of couple more minor comments: >>>>>>>>>>>>>> >>>>>>>>>>>>>> LM4. Regarding the new “protocol” field added to >>>>>>> MemberDescription. >>>>>>>>>>>> Should >>>>>>>>>>>>>> we consider reusing the existing GroupProtocol enum instead of >>>>>>>>> String? >>>>>>>>>>>>>> (It’s the one we use from the consumer side to refer to the >>>>>>>> protocol >>>>>>>>> in >>>>>>>>>>>>>> use, just missing Share I notice). >>>>>>>>>>>>>> >>>>>>>>>>>>>> LM5. Regarding the change to the output for >>>>>>> kafka-consumer-groups, >>>>>>>>> the >>>>>>>>>>>>>> command shown does not include the —state option, but the >>>> output >>>>>>>>> shows >>>>>>>>>>>>>> state info (state, #members, epochs). I would guess that we >>>> want >>>>>>> to >>>>>>>>>>>> modify >>>>>>>>>>>>>> the output only when we describe a group with the —state >>>> —verbose >>>>>>>>>>>> option, >>>>>>>>>>>>>> is my understanding right? If my understanding is right we’re >>>>>>> just >>>>>>>>>>>> missing >>>>>>>>>>>>>> adding the —state in the example, and the KIP does not >>>> introduce >>>>>>>> any >>>>>>>>>>>>>> changes to the —describe —verbose option. (If not, it would >>>> mean >>>>>>> a >>>>>>>>>>>> bigger >>>>>>>>>>>>>> change to the output of —describe —verbose which I expect is >>>> not >>>>>>>> the >>>>>>>>>>>>>> intention?) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>> >>>>>>>>>>>>>> Lianet >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Nov 20, 2024 at 2:12 AM PoAn Yang <yangp...@gmail.com >>> >>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Chia-Ping, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for the review and suggestions. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> CI0: Thanks for the reminder. Update validVersions in >>>>>>>>>>>>>>> ConsumerGroupDescribeRequest to 0-1. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> CI1: Yes, I use `ConsumerGroupMember#useClassicProtocol` to >>>>>>> check >>>>>>>>>>>> whether >>>>>>>>>>>>>>> a member in consumer group uses “classic” or “consumer” >>>>>>> protocol. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> CI2: Yes, a member in share group always uses “share” >>>> protocol. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> CI3: Add a table to show meaning of “classic”, “consumer”, >> and >>>>>>>>> “share” >>>>>>>>>>>>>>> protocol. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> BTW, the vote thread is in >>>>>>>>>>>>>>> >>>>>>> https://lists.apache.org/thread/rb25tf75tzf4c7jqqldlo5jh9w8chsq6. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> PoAn >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Nov 20, 2024, at 11:46 AM, Chia-Ping Tsai < >>>>>>>> chia7...@apache.org> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> hi PoAn >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> CI0: We have to bump the version of >>>>>>> ConsumerGroupDescribeRequest >>>>>>>>> as >>>>>>>>>>>>>>> well, so server can distinguish the new and old behavior. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> CI1: the type of new filed is string, so I guess you plan to >>>>>>> use >>>>>>>>>>>>>>> `ConsumerGroupMember#useClassicProtocol` [0] flag to return >>>>>>> either >>>>>>>>>>>>>>> "classic" or "consumer", right? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> CI2: `MemberDescription` is used by `ShareGroupDescription` >>>>>>> too, >>>>>>>> so >>>>>>>>>>>> the >>>>>>>>>>>>>>> new filed protocol in shared group is always "shared", right? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> CI3: Could you consider adding a table to show the value of >>>> the >>>>>>>>>>>> protocol >>>>>>>>>>>>>>> field in each case? Andrew has a beautiful table in KIP-1043 >>>>>>> that >>>>>>>>>>>> lists all >>>>>>>>>>>>>>> possible protocol names. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [0] >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >> https://github.com/apache/kafka/blob/441a6d0b790f4a17b454caeea7588a6b90fbd9db/group-coordinator/src/main/java/org/apache/kafka/coordinator/group/modern/consumer/ConsumerGroupMember.java#L454 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>> Chia-Ping >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 2024/11/19 15:45:16 PoAn Yang wrote: >>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks for the reminder. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> DJ4: Add related content to the KIP. >>>>>>>>>>>>>>>>> Bump validVersions in ConsumerGroupDescribeResponse to >>>> “0-1”. >>>>>>>>>>>>>>>>> Add a new field “Protocol” to >> ConsumerGroupDescribeResponse. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> PoAn >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Nov 19, 2024, at 3:57 PM, David Jacot >>>>>>>>>>>> <dja...@confluent.io.INVALID> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi PoAn, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> DJ4: Could you please also include the required changes to >>>>>>> the >>>>>>>>>>>>>>>>>> ConsumerGroupDescribed API in the public interface >> section? >>>>>>> We >>>>>>>>>>>>>>> basically >>>>>>>>>>>>>>>>>> need to bump the version, add the new field, etc. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Mon, Nov 18, 2024 at 5:40 AM PoAn Yang < >>>>>>> yangp...@gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> DJ4: Add a new field protocol to MemberDescription, >>>>>>>>>>>>>>>>>>> so the command line tool can show protocol information >>>> when >>>>>>>>> users >>>>>>>>>>>>>>> describe >>>>>>>>>>>>>>>>>>> members. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> If there is no further suggestion, I will start a vote >>>>>>> thread >>>>>>>>>>>> today. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> PoAn >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Nov 15, 2024, at 11:50 PM, David Jacot >>>>>>>>>>>>>>> <dja...@confluent.io.INVALID> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> DJ2: Using "-" sounds good to me. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> DJ3: That seems reasonable to me. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> DJ4: Why not add it right now? I don't want to change >> the >>>>>>>>> output >>>>>>>>>>>> of >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>> tool too many times. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>> David >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Fri, Nov 15, 2024 at 3:23 PM PoAn Yang < >>>>>>>> yangp...@gmail.com> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hi David / Andrew, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks for review. Thanks Andrew for picking up >>>>>>>>>>>>>>> kafka-share-groups.sh >>>>>>>>>>>>>>>>>>>>> implementation. >>>>>>>>>>>>>>>>>>>>> I will handle kafka-consumer-groups.sh. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> DJ3: After discussing with @Chia-Ping Tsai, we think >>>> that >>>>>>>>> using >>>>>>>>>>>> new >>>>>>>>>>>>>>>>>>> format >>>>>>>>>>>>>>>>>>>>> is more clear. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> The new format will be like >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >> <topic-name1>:<partition-id1>,<partition-id2>;<topic-name2>:<partition-id1>,<partition-id2> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Using colon(:) to concat topic name and partition IDs. >>>>>>>>>>>>>>>>>>>>> Using comma(,) to concat partition IDs within same >> topic >>>>>>>> name. >>>>>>>>>>>>>>>>>>>>> Using semicolon(;) to concat topic strings. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> AS3: Fix it with kafka-share-groups.sh. Thanks. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> If there is no further suggestion, I will start a vote >>>>>>>> thread >>>>>>>>>>>> next >>>>>>>>>>>>>>>>>>> Monday. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> PoAn >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On Nov 14, 2024, at 9:05 PM, Andrew Schofield < >>>>>>>>>>>>>>>>>>>>> andrew_schofield_j...@outlook.com> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi PoAn, >>>>>>>>>>>>>>>>>>>>>> DJ2: I was just going to comment that "-" would be a >>>> more >>>>>>>>>>>>>>> appropriate >>>>>>>>>>>>>>>>>>>>> missing value, but >>>>>>>>>>>>>>>>>>>>>> you got there first. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> AS3: The examples for kafka-share-groups.sh include >>>>>>>>>>>>>>>>>>>>> kafka-consumer-groups.sh in the >>>>>>>>>>>>>>>>>>>>>> command line. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> If this is accepted in time, I'm happy to pick up the >>>>>>>>>>>>>>> implementation of >>>>>>>>>>>>>>>>>>>>> the share groups >>>>>>>>>>>>>>>>>>>>>> part of this if it helps. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> Andrew >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ________________________________________ >>>>>>>>>>>>>>>>>>>>>> From: Frank Yang <yangp...@gmail.com> >>>>>>>>>>>>>>>>>>>>>> Sent: 14 November 2024 10:48 >>>>>>>>>>>>>>>>>>>>>> To: dev@kafka.apache.org <dev@kafka.apache.org> >>>>>>>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] KIP-1099: Extend >>>>>>>> kafka-consumer-groups >>>>>>>>>>>>>>> command >>>>>>>>>>>>>>>>>>>>> line tool to support new consumer group >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi David, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks for the review and suggestion! I would like to >>>> get >>>>>>>>> this >>>>>>>>>>>> in >>>>>>>>>>>>>>> AK >>>>>>>>>>>>>>>>>>> 4.0 >>>>>>>>>>>>>>>>>>>>> as well. I will do my best. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> DJ1: Update KIP to put GROUP-EPOCH and >>>>>>>>> TARGET-ASSIGNMENT-EPOCH >>>>>>>>>>>>>>> before >>>>>>>>>>>>>>>>>>>>> #MEMBERS. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> DJ2: I prefer to follow current missing column value >>>> “-“. >>>>>>>>>>>>>>> (reference < >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >> https://github.com/apache/kafka/blob/18199028a672fd973ac37bf26316994babc2a6da/tools/src/main/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommand.java#L92 >>>>>>>>>>>>>>>>>>>>>> ) >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> DJ3: Update KIP to use CURRENT-EPOCH >> CURRENT-ASSIGNMENT >>>>>>>>>>>>>>> TARGET-EPOCH >>>>>>>>>>>>>>>>>>>>> TARGET-ASSIGNMENT. >>>>>>>>>>>>>>>>>>>>>> Remove GROUP-EPOCH. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> For assignment value, it follows current output >>>>>>> (reference >>>>>>>> < >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >> https://github.com/apache/kafka/blob/18199028a672fd973ac37bf26316994babc2a6da/tools/src/main/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommand.java#L413-L418 >>>>>>>>>>>>>>>>>>>> ). >>>>>>>>>>>>>>>>>>>>> I think the form of `topicid-partitionid` is more >> clear. >>>>>>>>>>>>>>>>>>>>>> If we would like to use this form, I will update both >>>>>>>> output >>>>>>>>> in >>>>>>>>>>>>>>>>>>>>> kafka-consumer-groups.sh and kafka-share-groups.sh. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> DJ4: It looks like DescribeGroupsResponseData only has >>>>>>>>> protocol >>>>>>>>>>>>>>> type at >>>>>>>>>>>>>>>>>>>>> group level. >>>>>>>>>>>>>>>>>>>>>> Both DescribeGroupsResponseData and >>>>>>>>>>>>>>> ConsumerGroupDescribeResponseData >>>>>>>>>>>>>>>>>>>>> don’t have protocol at member level. >>>>>>>>>>>>>>>>>>>>>> Could we use a followup to add it? >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> DJ5: Update KIP to put LEADER-EPOCH before >>>>>>> CURRENT-OFFSET. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>> PoAn >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Nov 14, 2024, at 3:43 PM, David Jacot >>>>>>>>>>>>>>> <dja...@confluent.io.INVALID >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Hi PoAn, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP! I have a few minor >>>>>>>> comments/suggestions: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> DJ1: In the output of `--describe --verbose`, I would >>>>>>>>> suggest >>>>>>>>>>>>>>> putting >>>>>>>>>>>>>>>>>>>>>>> `GROUP-EPOCH` and `TARGET-ASSIGNMENT-EPOCH` before >>>>>>>>> `#MEMBERS`. >>>>>>>>>>>>>>>>>>>>>>> DJ2: Continuing on the above, I assume that we will >>>>>>> print >>>>>>>>> out >>>>>>>>>>>> N/A >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>> fields not supported by classic groups. Is this >>>> correct? >>>>>>>>>>>>>>>>>>>>>>> DJ3: In the output of `--describe --members >>>> --verbose`, >>>>>>> I >>>>>>>>>>>> wonder >>>>>>>>>>>>>>> if we >>>>>>>>>>>>>>>>>>>>>>> should use the following order and terms: >>>> CURRENT-EPOCH >>>>>>>>>>>>>>>>>>>>> CURRENT-ASSIGNMENT >>>>>>>>>>>>>>>>>>>>>>> TARGET-EPOCH TARGET-ASSIGNMENT . I would remove the >>>>>>>>> GROUP-EPOCH >>>>>>>>>>>>>>>>>>> because >>>>>>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>>>>> is already in the group description. The value `(0)` >>>>>>> used >>>>>>>>> for >>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>> assignment is incorrect. Here I suppose that we will >>>>>>> print >>>>>>>>> out >>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> list >>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>> partitions in the form of `topicid-partitionid`. >>>>>>>>>>>>>>>>>>>>>>> DJ4: Continuing on the above, I wonder if we should >>>> also >>>>>>>> add >>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>> protocol >>>>>>>>>>>>>>>>>>>>>>> used `classic` or `consumer`. For context, it is >>>>>>> possible >>>>>>>> to >>>>>>>>>>>> have >>>>>>>>>>>>>>>>>>>>> `classic` >>>>>>>>>>>>>>>>>>>>>>> members and `consumer` members in a `consumer` group >>>>>>>> during >>>>>>>>> an >>>>>>>>>>>>>>> online >>>>>>>>>>>>>>>>>>>>>>> upgrade from the classic protocol to the consumer >>>>>>>> protocol. >>>>>>>>>>>> Having >>>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>>>>>>> information may be handy for administrators. What do >>>> you >>>>>>>>> think? >>>>>>>>>>>>>>>>>>>>>>> DJ5: In the output of `--describe --offsets >>>> --verbose`, >>>>>>> I >>>>>>>>> would >>>>>>>>>>>>>>>>>>> suggest >>>>>>>>>>>>>>>>>>>>>>> putting `LEADER-EPOCH` closer to `CURRENT-OFFSET`. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> It would be great if we could get this one in AK 4.0 >>>> as >>>>>>> it >>>>>>>>>>>>>>> changes the >>>>>>>>>>>>>>>>>>>>>>> output of the command. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>> DJ >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Fri, Nov 1, 2024 at 7:40 AM Frank Yang < >>>>>>>>> yangp...@gmail.com> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Sean / Andrew / Lianet, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks for all your review and suggestions. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> AS1, LM1, LM4: Change to add KIP-848 information >> when >>>>>>>> users >>>>>>>>>>>> give >>>>>>>>>>>>>>>>>>>>> —verbose >>>>>>>>>>>>>>>>>>>>>>>> option. >>>>>>>>>>>>>>>>>>>>>>>> —describe —verbose: shows group epoch and target >>>>>>>> assignment >>>>>>>>>>>>>>> epoch. >>>>>>>>>>>>>>>>>>>>>>>> —describe —members —verbose: shows above >> information, >>>>>>>>> member >>>>>>>>>>>>>>> epoch, >>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>> target assignment. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> AS2: Change to use MEMBER-EPOCH to align with >> KIP-848 >>>>>>>>>>>> definition. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> LM2: For classic group, it doesn’t have epoch, so I >>>> use >>>>>>>>>>>> Optional >>>>>>>>>>>>>>>>>>>>> fields in >>>>>>>>>>>>>>>>>>>>>>>> ConsumerGroupDescription. >>>>>>>>>>>>>>>>>>>>>>>> For share group, it relies on KIP-848. It must have >>>>>>>> epoch, >>>>>>>>> so >>>>>>>>>>>> I >>>>>>>>>>>>>>> use >>>>>>>>>>>>>>>>>>> int >>>>>>>>>>>>>>>>>>>>>>>> fields in ShareGroupDescription. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> LM3: Remove —state change. We can get group level >>>>>>>>> information >>>>>>>>>>>> by >>>>>>>>>>>>>>>>>>>>> —describe >>>>>>>>>>>>>>>>>>>>>>>> —verbose. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> SQ1: Add LEADER-EPOCH when users give —describe >>>>>>> —offsets >>>>>>>>>>>>>>> —verbose. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>>>>>>>>>>>> PoAn >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Nov 1, 2024, at 5:08 AM, Lianet M. < >>>>>>>> liane...@gmail.com >>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hello Frank, thanks for the KIP! A few comments: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> LM1. I strongly agree with Andrew's suggestion of >>>>>>> moving >>>>>>>>> this >>>>>>>>>>>>>>> into a >>>>>>>>>>>>>>>>>>>>>>>>> --verbose option. I expect someone would only >>>> attempt >>>>>>> to >>>>>>>>> make >>>>>>>>>>>>>>> sense >>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>> epochs while debugging an issue, not while trying >> to >>>>>>>> view >>>>>>>>> the >>>>>>>>>>>>>>> group >>>>>>>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>>>>>>>>>> member's info. (Also member-epoch makes more sense >>>> to >>>>>>>> me, >>>>>>>>>>>> even >>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>>>>> if we >>>>>>>>>>>>>>>>>>>>>>>>> end up agreeing in a --verbose). Related to both >>>>>>> issues, >>>>>>>>> my >>>>>>>>>>>>>>> take is >>>>>>>>>>>>>>>>>>>>> that >>>>>>>>>>>>>>>>>>>>>>>>> whoever is close to the protocol will expect >>>>>>>> member-epoch, >>>>>>>>>>>>>>> whoever >>>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>>> will probably won't even need to see the epochs at >>>>>>> all. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> LM2. Why are the epochs defined as Optional (don't >>>> we >>>>>>>>> expect >>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>> always >>>>>>>>>>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>>>>>>>>> them? with 0 initially, defined on the broker side >>>> for >>>>>>>> the >>>>>>>>>>>> group >>>>>>>>>>>>>>>>>>> ones, >>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>> client side for the member) >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> LM3. Why is the KIP including the –state option in >>>> the >>>>>>>>>>>> proposed >>>>>>>>>>>>>>>>>>>>> changes? >>>>>>>>>>>>>>>>>>>>>>>> ( >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=327977411#KIP1099:Extendkafkaconsumergroupscommandlinetooltosupportnewconsumergroup---state >>>>>>>>>>>>>>>>>>>>>>>>> ). >>>>>>>>>>>>>>>>>>>>>>>>> I get that the output would change in that example, >>>>>>> but >>>>>>>>> it’s >>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>> because >>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>> any change to the –state option itself. It's >> because >>>>>>> of >>>>>>>>> the >>>>>>>>>>>>>>> change >>>>>>>>>>>>>>>>>>>>>>>> proposed >>>>>>>>>>>>>>>>>>>>>>>>> to the –described (required with the --state), and >>>> the >>>>>>>>>>>> changes >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>>>> --describe are already explained above (seems >>>>>>> confusing, >>>>>>>>> got >>>>>>>>>>>> me >>>>>>>>>>>>>>>>>>>>> looking >>>>>>>>>>>>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>>>>>>>> a change to the state filter which seemed >>>> unrelated). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> LM4. Group epoch and target assignment epoch are >>>>>>>>> conceptually >>>>>>>>>>>>>>> at the >>>>>>>>>>>>>>>>>>>>>>>> group >>>>>>>>>>>>>>>>>>>>>>>>> level. vs member epoch that lands at a member >> level. >>>>>>> So >>>>>>>>>>>> wonder >>>>>>>>>>>>>>> if we >>>>>>>>>>>>>>>>>>>>>>>> should >>>>>>>>>>>>>>>>>>>>>>>>> show them accordingly (ex. using the --verbose >>>> option) >>>>>>>>>>>>>>>>>>>>>>>>> –describe –verbose => shows group epoch and target >>>>>>>>> assignment >>>>>>>>>>>>>>> epoch >>>>>>>>>>>>>>>>>>>>>>>>> –describe –members –verbose => shows member >>>>>>>> epoch >>>>>>>>>>>>>>> (along >>>>>>>>>>>>>>>>>>>>>>>>> with group epoch and target assignment epoch) >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Lianet >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 31, 2024 at 7:30 AM Andrew Schofield < >>>>>>>>>>>>>>>>>>>>>>>>> andrew_schofield_j...@outlook.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi PoAn, >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP. I have a few comments. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> AS1: It seems to me that these new additions are >>>> most >>>>>>>>>>>> useful to >>>>>>>>>>>>>>>>>>>>> people >>>>>>>>>>>>>>>>>>>>>>>>>> trying to understand >>>>>>>>>>>>>>>>>>>>>>>>>> the progress of rebalancing in quite some detail. >>>> The >>>>>>>>>>>>>>> information >>>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>>>>>>> really not understandable >>>>>>>>>>>>>>>>>>>>>>>>>> for most users who do not have deep knowledge of >>>>>>>>>>>> KIP-848/932. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> As a result, I suggest for kafka-share-groups.sh >>>> that >>>>>>>> you >>>>>>>>>>>> add a >>>>>>>>>>>>>>>>>>>>>>>> --members >>>>>>>>>>>>>>>>>>>>>>>>>> --verbose option >>>>>>>>>>>>>>>>>>>>>>>>>> and only include the new information in the output >>>>>>> for >>>>>>>>> that >>>>>>>>>>>>>>> option, >>>>>>>>>>>>>>>>>>>>>>>> rather >>>>>>>>>>>>>>>>>>>>>>>>>> than changing the >>>>>>>>>>>>>>>>>>>>>>>>>> non-verbose --members output. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I also make a similar suggestion for >>>>>>>>>>>> kafka-consumer-groups.sh >>>>>>>>>>>>>>>>>>>>> --members >>>>>>>>>>>>>>>>>>>>>>>>>> and only add the >>>>>>>>>>>>>>>>>>>>>>>>>> new information for the --verbose output. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> AS2: I strongly suggest that you use MEMBER-EPOCH >>>>>>>>> instead of >>>>>>>>>>>>>>>>>>>>>>>>>> CONSUMER-EPOCH. >>>>>>>>>>>>>>>>>>>>>>>>>> I think it's more confusing not following the >>>>>>>>> terminology of >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>> KIPs. >>>>>>>>>>>>>>>>>>>>>>>> For >>>>>>>>>>>>>>>>>>>>>>>>>> one thing, >>>>>>>>>>>>>>>>>>>>>>>>>> we already have "member" in the admin client such >>>> as >>>>>>>>>>>>>>>>>>>>> MemberDescription. >>>>>>>>>>>>>>>>>>>>>>>>>> It's not a >>>>>>>>>>>>>>>>>>>>>>>>>> new term. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> Andrew >>>>>>>>>>>>>>>>>>>>>>>>>> ________________________________________ >>>>>>>>>>>>>>>>>>>>>>>>>> From: PoAn Yang <pay...@apache.org> >>>>>>>>>>>>>>>>>>>>>>>>>> Sent: 25 October 2024 13:55 >>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@kafka.apache.org <dev@kafka.apache.org> >>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] KIP-1099: Extend >>>>>>>>>>>> kafka-consumer-groups >>>>>>>>>>>>>>>>>>> command >>>>>>>>>>>>>>>>>>>>>>>> line >>>>>>>>>>>>>>>>>>>>>>>>>> tool to support new consumer group >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Lucas, >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks for the review! >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 1) Yes, I add related change for >>>>>>> kafka-share-groups.sh >>>>>>>> to >>>>>>>>>>>> the >>>>>>>>>>>>>>> KIP. >>>>>>>>>>>>>>>>>>>>> Could >>>>>>>>>>>>>>>>>>>>>>>>>> you take a look? Thanks for the suggestion. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> 2) We use CONSUMER-ID as member ID. If we use >>>>>>>>> MEMBER-EPOCH >>>>>>>>>>>>>>> here, >>>>>>>>>>>>>>>>>>>>> users >>>>>>>>>>>>>>>>>>>>>>>> may >>>>>>>>>>>>>>>>>>>>>>>>>> confuse what is different between CONSUMER and >>>>>>> MEMBER. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>>> PoAn >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On 2024/10/23 13:28:17 Lucas Brutschy wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Frank, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> thanks for the KIP! >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 1) For consistency, should we do the same for >>>>>>>>>>>>>>>>>>>>>>>>>>> kafka-share-groups.sh, ShareGroupDescription, >>>> etc. ? >>>>>>>>> Even >>>>>>>>>>>> if >>>>>>>>>>>>>>> we do >>>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>>>>> implement it right now if the share group >>>>>>>> implementation >>>>>>>>>>>> may >>>>>>>>>>>>>>> still >>>>>>>>>>>>>>>>>>>>> be >>>>>>>>>>>>>>>>>>>>>>>>>>> incomplete, it may make sense to include it in >> the >>>>>>>> KIP. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> 2) Why call it CONSUMER-EPOCH, not MEMBER-EPOCH? >>>>>>> That >>>>>>>>> would >>>>>>>>>>>>>>> seem >>>>>>>>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>>>>>>>>>>> consistent. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>>>>>>> Lucas >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 23, 2024 at 2:41 PM Frank Yang < >>>>>>>>>>>>>>> yangp...@gmail.com> >>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I would like to kick off the discussion of >>>>>>> KIP-1099. >>>>>>>>> This >>>>>>>>>>>> KIP >>>>>>>>>>>>>>>>>>>>> enhances >>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>>> kafka-consumer-groups tools to include state >>>> which >>>>>>> is >>>>>>>>>>>>>>> introduced >>>>>>>>>>>>>>>>>>> by >>>>>>>>>>>>>>>>>>>>>>>>>> KIP-848. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> KIP-1099: Extend kafka-consumer-groups command >>>> line >>>>>>>>> tool >>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>> support >>>>>>>>>>>>>>>>>>>>>>>> new >>>>>>>>>>>>>>>>>>>>>>>>>>>> consumer group - Apache Kafka - Apache Software >>>>>>>>> Foundation >>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org >>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> [image: favicon.ico] >>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> < >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you, >>>>>>>>>>>>>>>>>>>>>>>>>>>> PoAn >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>>>> >>>> >> >> >>
