hi PoAn `isClassic` is good to me :)
Best, Chia-Ping PoAn Yang <yangp...@gmail.com> 於 2024年11月23日 週六 下午12:35寫道： > Hi Chia-Ping / David, > > Thanks for the great suggestion. > > How about using “isClassic”? > The term like “isLegacy” may be equivocal in the future. > For example, if we have a newer version of consumer / share consumer, the > meaning will be different for different cases. > If we use “isClassic”, for MemberDescription in ClassicGroupDescription, > the value can be always “true”. > In ConsumerGroupDescription, it’s “true” when it’s a classic member. > In ShareGroupDescription, it’s always “false”. > > Thanks, > PoAn > > > On Nov 23, 2024, at 1:02 AM, Chia-Ping Tsai <chia7...@gmail.com> wrote: > > > > hi DJ > > > >> My goal was to have a way to tell the user that his (new) consumer group > > may have non-upgraded members yet. I wonder if we could use a boolean to > > signal this, e.g. isLegacy. Then, in the CLI we could have a column which > > is only displayed when they are non upgraded members. What do you think? > > > > Yes, this is a good approach, but the downside is that the new isLegacy > > field in MemberDescription would be irrelevant for classic and shared > > groups. This would require thorough documentation to clarify its purpose. > > > > Additionally, we could use a boolean instead of a string for the new > field > > in the RPC to reduce the request size. > > Best, > > Chia-Ping > > > > > > > > David Jacot <dja...@confluent.io.invalid> 於 2024年11月23日 週六 上午12:41寫道： > > > >> Hi both, > >> > >> Ah, I understand what you mean now. You're saying that we have two > protocol > >> fields. One at the group level and one at the member level (the new > one). > >> The two fields are actually two different things. The former is the > name of > >> the embedded protocol used in the classic protocol whereas the latter is > >> the name of the protocol/api used. This is indeed confusing. > >> > >> My goal was to have a way to tell the user that his (new) consumer group > >> may have non-upgraded members yet. I wonder if we could use a boolean to > >> signal this, e.g. isLegacy. Then, in the CLI we could have a column > which > >> is only displayed when they are non upgraded members. What do you think? > >> > >> If we cannot find a good way to represent this, we could drop it from > this > >> KIP too. > >> > >> Best, > >> David > >> > >> On Fri, Nov 22, 2024 at 5:12 PM PoAn Yang <yangp...@gmail.com> wrote: > >> > >>> Hi Chia-Ping / David, > >>> > >>> Thanks for the review and suggestions. > >>> > >>>>> I don't fully follow your comment so please excuse me if I say > >> something > >>>>> wrong. > >>> > >>> I think Chia-Ping wants to say: originally, the default protocol in > >>> classic group is “consumer”. > >>> After we have a new consumer group, the protocol of a classic member in > >>> consumer group is “classic”. > >>> The MemberDescription is used by both ClassicGroupDescription and > >>> ConsumerGroupDescription. > >>> If we add the protocol field to MemberDescription, the protocol of a > >>> classic member in classic group is “consumer”, but it’s “classic” in > >>> consumer group. > >>> To avoid misleading word, we can add a new collection of member > protocols > >>> to ConsumerGroupDescription only. > >>> The original goal is to show member protocol during consumer migration, > >> so > >>> we don’t need to add this information to ClassicGroupDescription. > >>> > >>> Please correct me if I misunderstood anything. > >>> > >>> Thanks, > >>> PoAn > >>> > >>>> On Nov 22, 2024, at 7:00 PM, Chia-Ping Tsai <chia7...@gmail.com> > >> wrote: > >>>> > >>>> hi DJ > >>>> > >>>>> How about we use another Jira to add “SHARE” to GroupProtocol and > >> change > >>>> return type of ModernGroup#protocolType? > >>>> > >>>> This is not my comment ... sorry for the incorrect quote :( > >>>> > >>>>> A member in a classic group should report "classic". > >>>> > >>>> I completely agree. However, in this case, > >>> ClassicGroupDescription#protocol > >>>> returns 'consumer', while ClassicGroupDescription#members[x].protocol > >>>> returns 'classic'. This inconsistency feels a bit strange to me. > >>>> > >>>> Best, > >>>> Chia-Ping > >>>> > >>>> > >>>> David Jacot <dja...@confluent.io.invalid> 於 2024年11月22日 週五 下午6:51寫道： > >>>> > >>>>> Hi Chia, > >>>>> > >>>>>>> LM4: I agree it’s better to use a type like GroupProtocol enum, but > >> I > >>>>> would like to align the format with kafka-groups.sh. > >>>>> How about we use another Jira to add “SHARE” to GroupProtocol and > >> change > >>>>> return type of ModernGroup#protocolType? > >>>>> > >>>>> My understanding is that `GroupProtocol` is used to drive the config > >> of > >>> the > >>>>> `Consumer`. Hence adding `Share` seems wrong to me. Or are you > >>> referring to > >>>>> another GroupProtocol enum? Perhaps GroupType enum? > >>>>> > >>>>>> In fact, I'm wondering if we should even add 'protocol' to > >>>>> MemberDescription, > >>>>> since the field has different values between classic and consumer > >>> groups—it > >>>>> shows 'consumer' in classic groups, whereas it shows 'classic' in > >>> consumer > >>>>> groups. Yes, it seems like a typo, but it's true. This inconsistency > >>> feels > >>>>> confusing to me. For example, users call Admin#describeConsumerGroups > >>> and > >>>>> see that a member has the 'consumer' protocol when it's in a classic > >>> group. > >>>>> Then, if the group is upgraded to a consumer group, calling > >>>>> Admin#describeConsumerGroups again shows the protocol as 'classic' > for > >>> the > >>>>> same member. > >>>>> > >>>>> I don't fully follow your comment so please excuse me if I say > >> something > >>>>> wrong. > >>>>> > >>>>> I think that it should work as follows: > >>>>> * A member in a classic group should report "classic". > >>>>> * A member using the consumer protocol in a consumer group should > >> report > >>>>> "consumer". > >>>>> * A member using the classic protocol in a consumer group should > >> report > >>>>> "classic". > >>>>> * A member in a share group should report "share". > >>>>> > >>>>>> If all we want to do is distinguish classic members in the > >>>>> ConsumerGroupDescription, then we shouldn't modify the common > >>>>> MemberDescription structure. Instead, we could add a collection to > >>>>> ConsumerGroupDescription to trace members using the classic protocol, > >>> which > >>>>> would allow us to enrich the output of command-line tools. > >>>>> > >>>>> Personally, I believe that using different collections per protocol > >>> types > >>>>> is not necessary. A field in the member is more than enough. > >>>>> > >>>>> Best, > >>>>> DJ > >>>>> > >>>>> On Fri, Nov 22, 2024 at 12:01 PM Chia-Ping Tsai <chia7...@gmail.com> > >>>>> wrote: > >>>>> > >>>>>> hi PoAn > >>>>>> > >>>>>>> LM4: I agree it’s better to use a type like GroupProtocol enum, but > >> I > >>>>>> would like to align the format with kafka-groups.sh. > >>>>>> How about we use another Jira to add “SHARE” to GroupProtocol and > >>> change > >>>>>> return type of ModernGroup#protocolType? > >>>>>> > >>>>>> MemberDescription is also used by ClassicGroupDescription, and in > >>> classic > >>>>>> groups, the protocol can be an arbitrary string, so we can't use an > >>> enum. > >>>>>> > >>>>>> In fact, I'm wondering if we should even add 'protocol' to > >>>>>> MemberDescription, > >>>>>> since the field has different values between classic and consumer > >>>>> groups—it > >>>>>> shows 'consumer' in classic groups, whereas it shows 'classic' in > >>>>> consumer > >>>>>> groups. Yes, it seems like a typo, but it's true. This inconsistency > >>>>> feels > >>>>>> confusing to me. For example, users call > Admin#describeConsumerGroups > >>> and > >>>>>> see that a member has the 'consumer' protocol when it's in a classic > >>>>> group. > >>>>>> Then, if the group is upgraded to a consumer group, calling > >>>>>> Admin#describeConsumerGroups again shows the protocol as 'classic' > >> for > >>>>> the > >>>>>> same member. > >>>>>> > >>>>>> If all we want to do is distinguish classic members in the > >>>>>> ConsumerGroupDescription, then we shouldn't modify the common > >>>>>> MemberDescription structure. Instead, we could add a collection to > >>>>>> ConsumerGroupDescription to trace members using the classic > protocol, > >>>>> which > >>>>>> would allow us to enrich the output of command-line tools. > >>>>>> Best, > >>>>>> Chia-Ping > >>>>>> > >>>>>> PoAn Yang <yangp...@gmail.com> 於 2024年11月22日 週五 下午3:46寫道： > >>>>>> > >>>>>>> Hi Lianet / Jeff, > >>>>>>> > >>>>>>> Thanks for the review. > >>>>>>> > >>>>>>> LM5: The kafka-share-groups.sh also uses “—describe” to show > offsets > >>>>>>> information. > >>>>>>> Change to use “—describe —state —verbose” to show group level > >>>>> information > >>>>>>> in kafka-share-groups.sh. > >>>>>>> > >>>>>>> LM6: Update the example. > >>>>>>> > >>>>>>> LM7: Add a description to mention “—verbose” is a new option in > >>>>>>> ShareGroupCommandOptions. > >>>>>>> > >>>>>>> JK2: Yes, add the statement about format change for ASSIGNMENT > >> value. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> PoAn > >>>>>>> > >>>>>>>> On Nov 22, 2024, at 7:35 AM, Jeff Kim <jeffkb...@apache.org> > >> wrote: > >>>>>>>> > >>>>>>>> Hi PoAn, > >>>>>>>> > >>>>>>>> JK2: Can we state in the KIP that we will reformat the existing > >>>>>>>> classic group's `--members --verbose` as well, specifically the > >>>>>>>> "ASSIGNMENT" column to include the topic name? > >>>>>>>> > >>>>>>>> Thanks! > >>>>>>>> > >>>>>>>> On 2024/11/21 14:33:37 "Lianet M." wrote: > >>>>>>>>> Thanks for the updates PoAn! > >>>>>>>>> > >>>>>>>>> LM6. Just a nit, under the --describe --state --verbose the > header > >>>>> is > >>>>>>> fine > >>>>>>>>> but the example is still missing the --state argument. > >>>>>>>>> > >>>>>>>>> LM7. Under the Proposed changes for kafka-share-groups.sh, should > >> we > >>>>>>>>> clearly mention that the KIP adds a new --verbose option? I think > >>>>> it's > >>>>>>>>> confusing because it's presented as if the option already exists > >> and > >>>>>>> we're > >>>>>>>>> only changing the output (which is the case for > >>>>> kafka-consumer-groups > >>>>>>> but > >>>>>>>>> not for kafka-share-groups) > >>>>>>>>> > >>>>>>>>> That's all on my side. Thanks! > >>>>>>>>> > >>>>>>>>> Lianet > >>>>>>>>> > >>>>>>>>> On Wed, Nov 20, 2024 at 11:12 PM PoAn Yang <yangp...@gmail.com> > >>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Hi Lianet / Jeff, > >>>>>>>>>> > >>>>>>>>>> Thanks for review and suggestions. > >>>>>>>>>> > >>>>>>>>>> LM4: At group level, the protocol field is from > >>>>>>> ModernGroup#protocolType. > >>>>>>>>>> It’s a string. > >>>>>>>>>> The value is defined in different places like > >>>>>> ShareGroup#PROTOCOL_TYPE > >>>>>>> and > >>>>>>>>>> ConsumerProtocol#PROTOCOL_TYPE. > >>>>>>>>>> In KIP-1043 < > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=305171038#KIP1043:Administrationofgroups-kafka-groups.sh > >>>>>>>> , > >>>>>>>>>> the kafka-groups.sh also shows protocol like “classic” / > >>>>> “consumer” / > >>>>>>>>>> “share”. > >>>>>>>>>> I agree it’s better to use a type like GroupProtocol enum, but I > >>>>>> would > >>>>>>>>>> like to align the format with kafka-groups.sh. > >>>>>>>>>> How about we use another Jira to add “SHARE” to GroupProtocol > and > >>>>>>> change > >>>>>>>>>> return type of ModernGroup#protocolType? > >>>>>>>>>> > >>>>>>>>>> LM5 / JK1: Sorry, I misunderstood LM3, so I deleted —state in > the > >>>>>>> title. > >>>>>>>>>> You’re correct. I would like to use `—describe —state —verbose` > >> to > >>>>>> show > >>>>>>>>>> group level information. > >>>>>>>>>> For `—describe —verbose`, it will show output as same as > >> `—describe > >>>>>>>>>> —offsets —verbose`, > >>>>>>>>>> because `—describe` also shows output as same as `—describe > >>>>>> —offsets`. > >>>>>>>>>> > >>>>>>>>>> JK2: Yes, I only use consumer group in the sample output, but > the > >>>>>>> classic > >>>>>>>>>> group will use the same format as well. > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> PoAn > >>>>>>>>>> > >>>>>>>>>>> On Nov 21, 2024, at 8:28 AM, Jeff Kim <jeffkb...@apache.org> > >>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Hi PoAn, > >>>>>>>>>>> > >>>>>>>>>>> Thanks for the KIP. I have some questions: > >>>>>>>>>>> > >>>>>>>>>>> JK1: (Realized this is similar to LM5) On `--describe > --verbose` > >>>>>>>>>> proposed changes - doesn't `--describe $group` default to > >> printing > >>>>>> the > >>>>>>>>>> offsets? Perhaps you're referring to the `--state` argument? > >> Also, > >>>>>>> would > >>>>>>>>>> that mean the default `--describe $group --verbose` command > would > >>>>> not > >>>>>>> print > >>>>>>>>>> the added field to `--offsets verbose` (leader-epoch) or would > >> it? > >>>>>>>>>>> > >>>>>>>>>>> JK2: Looking at ConsumerGroupCommand.java, the existing > >>>>> "ASSIGNMENT" > >>>>>>>>>> column under `--members --verbose` does group the topic > >> partitions > >>>>> by > >>>>>>> topic > >>>>>>>>>> but does not prefix the grouping with the topic name like you're > >>>>>>> proposal: > >>>>>>>>>> "my_topic:0,1;new_topic:0,1". Should we do apply the same format > >>>>> for > >>>>>>> the > >>>>>>>>>> classic group as well? > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> On 2024/11/20 21:32:41 "Lianet M." wrote: > >>>>>>>>>>>> Hello PoAn, just of couple more minor comments: > >>>>>>>>>>>> > >>>>>>>>>>>> LM4. Regarding the new “protocol” field added to > >>>>> MemberDescription. > >>>>>>>>>> Should > >>>>>>>>>>>> we consider reusing the existing GroupProtocol enum instead of > >>>>>>> String? > >>>>>>>>>>>> (It’s the one we use from the consumer side to refer to the > >>>>>> protocol > >>>>>>> in > >>>>>>>>>>>> use, just missing Share I notice). > >>>>>>>>>>>> > >>>>>>>>>>>> LM5. Regarding the change to the output for > >>>>> kafka-consumer-groups, > >>>>>>> the > >>>>>>>>>>>> command shown does not include the —state option, but the > >> output > >>>>>>> shows > >>>>>>>>>>>> state info (state, #members, epochs). I would guess that we > >> want > >>>>> to > >>>>>>>>>> modify > >>>>>>>>>>>> the output only when we describe a group with the —state > >> —verbose > >>>>>>>>>> option, > >>>>>>>>>>>> is my understanding right? If my understanding is right we’re > >>>>> just > >>>>>>>>>> missing > >>>>>>>>>>>> adding the —state in the example, and the KIP does not > >> introduce > >>>>>> any > >>>>>>>>>>>> changes to the —describe —verbose option. (If not, it would > >> mean > >>>>> a > >>>>>>>>>> bigger > >>>>>>>>>>>> change to the output of —describe —verbose which I expect is > >> not > >>>>>> the > >>>>>>>>>>>> intention?) > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks! > >>>>>>>>>>>> > >>>>>>>>>>>> Lianet > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Wed, Nov 20, 2024 at 2:12 AM PoAn Yang <yangp...@gmail.com > > > >>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Hi Chia-Ping, > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks for the review and suggestions. > >>>>>>>>>>>>> > >>>>>>>>>>>>> CI0: Thanks for the reminder. Update validVersions in > >>>>>>>>>>>>> ConsumerGroupDescribeRequest to 0-1. > >>>>>>>>>>>>> > >>>>>>>>>>>>> CI1: Yes, I use `ConsumerGroupMember#useClassicProtocol` to > >>>>> check > >>>>>>>>>> whether > >>>>>>>>>>>>> a member in consumer group uses “classic” or “consumer” > >>>>> protocol. > >>>>>>>>>>>>> > >>>>>>>>>>>>> CI2: Yes, a member in share group always uses “share” > >> protocol. > >>>>>>>>>>>>> > >>>>>>>>>>>>> CI3: Add a table to show meaning of “classic”, “consumer”, > and > >>>>>>> “share” > >>>>>>>>>>>>> protocol. > >>>>>>>>>>>>> > >>>>>>>>>>>>> BTW, the vote thread is in > >>>>>>>>>>>>> > >>>>> https://lists.apache.org/thread/rb25tf75tzf4c7jqqldlo5jh9w8chsq6. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> PoAn > >>>>>>>>>>>>> > >>>>>>>>>>>>>> On Nov 20, 2024, at 11:46 AM, Chia-Ping Tsai < > >>>>>> chia7...@apache.org> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> hi PoAn > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> CI0: We have to bump the version of > >>>>> ConsumerGroupDescribeRequest > >>>>>>> as > >>>>>>>>>>>>> well, so server can distinguish the new and old behavior. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> CI1: the type of new filed is string, so I guess you plan to > >>>>> use > >>>>>>>>>>>>> `ConsumerGroupMember#useClassicProtocol` [0] flag to return > >>>>> either > >>>>>>>>>>>>> "classic" or "consumer", right? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> CI2: `MemberDescription` is used by `ShareGroupDescription` > >>>>> too, > >>>>>> so > >>>>>>>>>> the > >>>>>>>>>>>>> new filed protocol in shared group is always "shared", right? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> CI3: Could you consider adding a table to show the value of > >> the > >>>>>>>>>> protocol > >>>>>>>>>>>>> field in each case? Andrew has a beautiful table in KIP-1043 > >>>>> that > >>>>>>>>>> lists all > >>>>>>>>>>>>> possible protocol names. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> [0] > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > https://github.com/apache/kafka/blob/441a6d0b790f4a17b454caeea7588a6b90fbd9db/group-coordinator/src/main/java/org/apache/kafka/coordinator/group/modern/consumer/ConsumerGroupMember.java#L454 > >>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>> Chia-Ping > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On 2024/11/19 15:45:16 PoAn Yang wrote: > >>>>>>>>>>>>>>> Hi David, > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks for the reminder. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> DJ4: Add related content to the KIP. > >>>>>>>>>>>>>>> Bump validVersions in ConsumerGroupDescribeResponse to > >> “0-1”. > >>>>>>>>>>>>>>> Add a new field “Protocol” to > ConsumerGroupDescribeResponse. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>> PoAn > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Nov 19, 2024, at 3:57 PM, David Jacot > >>>>>>>>>> <dja...@confluent.io.INVALID> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hi PoAn, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> DJ4: Could you please also include the required changes to > >>>>> the > >>>>>>>>>>>>>>>> ConsumerGroupDescribed API in the public interface > section? > >>>>> We > >>>>>>>>>>>>> basically > >>>>>>>>>>>>>>>> need to bump the version, add the new field, etc. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> David > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Mon, Nov 18, 2024 at 5:40 AM PoAn Yang < > >>>>> yangp...@gmail.com> > >>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Hi David, > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> DJ4: Add a new field protocol to MemberDescription, > >>>>>>>>>>>>>>>>> so the command line tool can show protocol information > >> when > >>>>>>> users > >>>>>>>>>>>>> describe > >>>>>>>>>>>>>>>>> members. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> If there is no further suggestion, I will start a vote > >>>>> thread > >>>>>>>>>> today. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>> PoAn > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Nov 15, 2024, at 11:50 PM, David Jacot > >>>>>>>>>>>>> <dja...@confluent.io.INVALID> > >>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Hi, > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> DJ2: Using "-" sounds good to me. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> DJ3: That seems reasonable to me. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> DJ4: Why not add it right now? I don't want to change > the > >>>>>>> output > >>>>>>>>>> of > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>> tool too many times. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>>>>> David > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Fri, Nov 15, 2024 at 3:23 PM PoAn Yang < > >>>>>> yangp...@gmail.com> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Hi David / Andrew, > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Thanks for review. Thanks Andrew for picking up > >>>>>>>>>>>>> kafka-share-groups.sh > >>>>>>>>>>>>>>>>>>> implementation. > >>>>>>>>>>>>>>>>>>> I will handle kafka-consumer-groups.sh. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> DJ3: After discussing with @Chia-Ping Tsai, we think > >> that > >>>>>>> using > >>>>>>>>>> new > >>>>>>>>>>>>>>>>> format > >>>>>>>>>>>>>>>>>>> is more clear. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> The new format will be like > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > <topic-name1>:<partition-id1>,<partition-id2>;<topic-name2>:<partition-id1>,<partition-id2> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Using colon(:) to concat topic name and partition IDs. > >>>>>>>>>>>>>>>>>>> Using comma(,) to concat partition IDs within same > topic > >>>>>> name. > >>>>>>>>>>>>>>>>>>> Using semicolon(;) to concat topic strings. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> AS3: Fix it with kafka-share-groups.sh. Thanks. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> If there is no further suggestion, I will start a vote > >>>>>> thread > >>>>>>>>>> next > >>>>>>>>>>>>>>>>> Monday. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>> PoAn > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On Nov 14, 2024, at 9:05 PM, Andrew Schofield < > >>>>>>>>>>>>>>>>>>> andrew_schofield_j...@outlook.com> wrote: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Hi PoAn, > >>>>>>>>>>>>>>>>>>>> DJ2: I was just going to comment that "-" would be a > >> more > >>>>>>>>>>>>> appropriate > >>>>>>>>>>>>>>>>>>> missing value, but > >>>>>>>>>>>>>>>>>>>> you got there first. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> AS3: The examples for kafka-share-groups.sh include > >>>>>>>>>>>>>>>>>>> kafka-consumer-groups.sh in the > >>>>>>>>>>>>>>>>>>>> command line. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> If this is accepted in time, I'm happy to pick up the > >>>>>>>>>>>>> implementation of > >>>>>>>>>>>>>>>>>>> the share groups > >>>>>>>>>>>>>>>>>>>> part of this if it helps. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>> Andrew > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> ________________________________________ > >>>>>>>>>>>>>>>>>>>> From: Frank Yang <yangp...@gmail.com> > >>>>>>>>>>>>>>>>>>>> Sent: 14 November 2024 10:48 > >>>>>>>>>>>>>>>>>>>> To: dev@kafka.apache.org <dev@kafka.apache.org> > >>>>>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] KIP-1099: Extend > >>>>>> kafka-consumer-groups > >>>>>>>>>>>>> command > >>>>>>>>>>>>>>>>>>> line tool to support new consumer group > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Hi David, > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks for the review and suggestion! I would like to > >> get > >>>>>>> this > >>>>>>>>>> in > >>>>>>>>>>>>> AK > >>>>>>>>>>>>>>>>> 4.0 > >>>>>>>>>>>>>>>>>>> as well. I will do my best. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> DJ1: Update KIP to put GROUP-EPOCH and > >>>>>>> TARGET-ASSIGNMENT-EPOCH > >>>>>>>>>>>>> before > >>>>>>>>>>>>>>>>>>> #MEMBERS. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> DJ2: I prefer to follow current missing column value > >> “-“. > >>>>>>>>>>>>> (reference < > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > https://github.com/apache/kafka/blob/18199028a672fd973ac37bf26316994babc2a6da/tools/src/main/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommand.java#L92 > >>>>>>>>>>>>>>>>>>>> ) > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> DJ3: Update KIP to use CURRENT-EPOCH > CURRENT-ASSIGNMENT > >>>>>>>>>>>>> TARGET-EPOCH > >>>>>>>>>>>>>>>>>>> TARGET-ASSIGNMENT. > >>>>>>>>>>>>>>>>>>>> Remove GROUP-EPOCH. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> For assignment value, it follows current output > >>>>> (reference > >>>>>> < > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > https://github.com/apache/kafka/blob/18199028a672fd973ac37bf26316994babc2a6da/tools/src/main/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommand.java#L413-L418 > >>>>>>>>>>>>>>>>>> ). > >>>>>>>>>>>>>>>>>>> I think the form of `topicid-partitionid` is more > clear. > >>>>>>>>>>>>>>>>>>>> If we would like to use this form, I will update both > >>>>>> output > >>>>>>> in > >>>>>>>>>>>>>>>>>>> kafka-consumer-groups.sh and kafka-share-groups.sh. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> DJ4: It looks like DescribeGroupsResponseData only has > >>>>>>> protocol > >>>>>>>>>>>>> type at > >>>>>>>>>>>>>>>>>>> group level. > >>>>>>>>>>>>>>>>>>>> Both DescribeGroupsResponseData and > >>>>>>>>>>>>> ConsumerGroupDescribeResponseData > >>>>>>>>>>>>>>>>>>> don’t have protocol at member level. > >>>>>>>>>>>>>>>>>>>> Could we use a followup to add it? > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> DJ5: Update KIP to put LEADER-EPOCH before > >>>>> CURRENT-OFFSET. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>> PoAn > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> On Nov 14, 2024, at 3:43 PM, David Jacot > >>>>>>>>>>>>> <dja...@confluent.io.INVALID > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Hi PoAn, > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Thanks for the KIP! I have a few minor > >>>>>> comments/suggestions: > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> DJ1: In the output of `--describe --verbose`, I would > >>>>>>> suggest > >>>>>>>>>>>>> putting > >>>>>>>>>>>>>>>>>>>>> `GROUP-EPOCH` and `TARGET-ASSIGNMENT-EPOCH` before > >>>>>>> `#MEMBERS`. > >>>>>>>>>>>>>>>>>>>>> DJ2: Continuing on the above, I assume that we will > >>>>> print > >>>>>>> out > >>>>>>>>>> N/A > >>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>> fields not supported by classic groups. Is this > >> correct? > >>>>>>>>>>>>>>>>>>>>> DJ3: In the output of `--describe --members > >> --verbose`, > >>>>> I > >>>>>>>>>> wonder > >>>>>>>>>>>>> if we > >>>>>>>>>>>>>>>>>>>>> should use the following order and terms: > >> CURRENT-EPOCH > >>>>>>>>>>>>>>>>>>> CURRENT-ASSIGNMENT > >>>>>>>>>>>>>>>>>>>>> TARGET-EPOCH TARGET-ASSIGNMENT . I would remove the > >>>>>>> GROUP-EPOCH > >>>>>>>>>>>>>>>>> because > >>>>>>>>>>>>>>>>>>> it > >>>>>>>>>>>>>>>>>>>>> is already in the group description. The value `(0)` > >>>>> used > >>>>>>> for > >>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>> assignment is incorrect. Here I suppose that we will > >>>>> print > >>>>>>> out > >>>>>>>>>> the > >>>>>>>>>>>>>>>>> list > >>>>>>>>>>>>>>>>>>> of > >>>>>>>>>>>>>>>>>>>>> partitions in the form of `topicid-partitionid`. > >>>>>>>>>>>>>>>>>>>>> DJ4: Continuing on the above, I wonder if we should > >> also > >>>>>> add > >>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> protocol > >>>>>>>>>>>>>>>>>>>>> used `classic` or `consumer`. For context, it is > >>>>> possible > >>>>>> to > >>>>>>>>>> have > >>>>>>>>>>>>>>>>>>> `classic` > >>>>>>>>>>>>>>>>>>>>> members and `consumer` members in a `consumer` group > >>>>>> during > >>>>>>> an > >>>>>>>>>>>>> online > >>>>>>>>>>>>>>>>>>>>> upgrade from the classic protocol to the consumer > >>>>>> protocol. > >>>>>>>>>> Having > >>>>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>>>>> information may be handy for administrators. What do > >> you > >>>>>>> think? > >>>>>>>>>>>>>>>>>>>>> DJ5: In the output of `--describe --offsets > >> --verbose`, > >>>>> I > >>>>>>> would > >>>>>>>>>>>>>>>>> suggest > >>>>>>>>>>>>>>>>>>>>> putting `LEADER-EPOCH` closer to `CURRENT-OFFSET`. > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> It would be great if we could get this one in AK 4.0 > >> as > >>>>> it > >>>>>>>>>>>>> changes the > >>>>>>>>>>>>>>>>>>>>> output of the command. > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>>> DJ > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> On Fri, Nov 1, 2024 at 7:40 AM Frank Yang < > >>>>>>> yangp...@gmail.com> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Hi Sean / Andrew / Lianet, > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Thanks for all your review and suggestions. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> AS1, LM1, LM4: Change to add KIP-848 information > when > >>>>>> users > >>>>>>>>>> give > >>>>>>>>>>>>>>>>>>> —verbose > >>>>>>>>>>>>>>>>>>>>>> option. > >>>>>>>>>>>>>>>>>>>>>> —describe —verbose: shows group epoch and target > >>>>>> assignment > >>>>>>>>>>>>> epoch. > >>>>>>>>>>>>>>>>>>>>>> —describe —members —verbose: shows above > information, > >>>>>>> member > >>>>>>>>>>>>> epoch, > >>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>> target assignment. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> AS2: Change to use MEMBER-EPOCH to align with > KIP-848 > >>>>>>>>>> definition. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> LM2: For classic group, it doesn’t have epoch, so I > >> use > >>>>>>>>>> Optional > >>>>>>>>>>>>>>>>>>> fields in > >>>>>>>>>>>>>>>>>>>>>> ConsumerGroupDescription. > >>>>>>>>>>>>>>>>>>>>>> For share group, it relies on KIP-848. It must have > >>>>>> epoch, > >>>>>>> so > >>>>>>>>>> I > >>>>>>>>>>>>> use > >>>>>>>>>>>>>>>>> int > >>>>>>>>>>>>>>>>>>>>>> fields in ShareGroupDescription. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> LM3: Remove —state change. We can get group level > >>>>>>> information > >>>>>>>>>> by > >>>>>>>>>>>>>>>>>>> —describe > >>>>>>>>>>>>>>>>>>>>>> —verbose. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> SQ1: Add LEADER-EPOCH when users give —describe > >>>>> —offsets > >>>>>>>>>>>>> —verbose. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> Thanks. > >>>>>>>>>>>>>>>>>>>>>> PoAn > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> On Nov 1, 2024, at 5:08 AM, Lianet M. < > >>>>>> liane...@gmail.com > >>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> Hello Frank, thanks for the KIP! A few comments: > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> LM1. I strongly agree with Andrew's suggestion of > >>>>> moving > >>>>>>> this > >>>>>>>>>>>>> into a > >>>>>>>>>>>>>>>>>>>>>>> --verbose option. I expect someone would only > >> attempt > >>>>> to > >>>>>>> make > >>>>>>>>>>>>> sense > >>>>>>>>>>>>>>>>> of > >>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>> epochs while debugging an issue, not while trying > to > >>>>>> view > >>>>>>> the > >>>>>>>>>>>>> group > >>>>>>>>>>>>>>>>> or > >>>>>>>>>>>>>>>>>>>>>>> member's info. (Also member-epoch makes more sense > >> to > >>>>>> me, > >>>>>>>>>> even > >>>>>>>>>>>>> more > >>>>>>>>>>>>>>>>>>> if we > >>>>>>>>>>>>>>>>>>>>>>> end up agreeing in a --verbose). Related to both > >>>>> issues, > >>>>>>> my > >>>>>>>>>>>>> take is > >>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>> whoever is close to the protocol will expect > >>>>>> member-epoch, > >>>>>>>>>>>>> whoever > >>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>> not > >>>>>>>>>>>>>>>>>>>>>>> will probably won't even need to see the epochs at > >>>>> all. > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> LM2. Why are the epochs defined as Optional (don't > >> we > >>>>>>> expect > >>>>>>>>>> to > >>>>>>>>>>>>>>>>> always > >>>>>>>>>>>>>>>>>>>>>> have > >>>>>>>>>>>>>>>>>>>>>>> them? with 0 initially, defined on the broker side > >> for > >>>>>> the > >>>>>>>>>> group > >>>>>>>>>>>>>>>>> ones, > >>>>>>>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>>> client side for the member) > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> LM3. Why is the KIP including the –state option in > >> the > >>>>>>>>>> proposed > >>>>>>>>>>>>>>>>>>> changes? > >>>>>>>>>>>>>>>>>>>>>> ( > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=327977411#KIP1099:Extendkafkaconsumergroupscommandlinetooltosupportnewconsumergroup---state > >>>>>>>>>>>>>>>>>>>>>>> ). > >>>>>>>>>>>>>>>>>>>>>>> I get that the output would change in that example, > >>>>> but > >>>>>>> it’s > >>>>>>>>>> not > >>>>>>>>>>>>>>>>>>> because > >>>>>>>>>>>>>>>>>>>>>> of > >>>>>>>>>>>>>>>>>>>>>>> any change to the –state option itself. It's > because > >>>>> of > >>>>>>> the > >>>>>>>>>>>>> change > >>>>>>>>>>>>>>>>>>>>>> proposed > >>>>>>>>>>>>>>>>>>>>>>> to the –described (required with the --state), and > >> the > >>>>>>>>>> changes > >>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>> --describe are already explained above (seems > >>>>> confusing, > >>>>>>> got > >>>>>>>>>> me > >>>>>>>>>>>>>>>>>>> looking > >>>>>>>>>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>>>>>>> a change to the state filter which seemed > >> unrelated). > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> LM4. Group epoch and target assignment epoch are > >>>>>>> conceptually > >>>>>>>>>>>>> at the > >>>>>>>>>>>>>>>>>>>>>> group > >>>>>>>>>>>>>>>>>>>>>>> level. vs member epoch that lands at a member > level. > >>>>> So > >>>>>>>>>> wonder > >>>>>>>>>>>>> if we > >>>>>>>>>>>>>>>>>>>>>> should > >>>>>>>>>>>>>>>>>>>>>>> show them accordingly (ex. using the --verbose > >> option) > >>>>>>>>>>>>>>>>>>>>>>> –describe –verbose => shows group epoch and target > >>>>>>> assignment > >>>>>>>>>>>>> epoch > >>>>>>>>>>>>>>>>>>>>>>> –describe –members –verbose => shows member > >>>>>> epoch > >>>>>>>>>>>>> (along > >>>>>>>>>>>>>>>>>>>>>>> with group epoch and target assignment epoch) > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> Thanks! > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> Lianet > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> On Thu, Oct 31, 2024 at 7:30 AM Andrew Schofield < > >>>>>>>>>>>>>>>>>>>>>>> andrew_schofield_j...@outlook.com> wrote: > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Hi PoAn, > >>>>>>>>>>>>>>>>>>>>>>>> Thanks for the KIP. I have a few comments. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> AS1: It seems to me that these new additions are > >> most > >>>>>>>>>> useful to > >>>>>>>>>>>>>>>>>>> people > >>>>>>>>>>>>>>>>>>>>>>>> trying to understand > >>>>>>>>>>>>>>>>>>>>>>>> the progress of rebalancing in quite some detail. > >> The > >>>>>>>>>>>>> information > >>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>> really not understandable > >>>>>>>>>>>>>>>>>>>>>>>> for most users who do not have deep knowledge of > >>>>>>>>>> KIP-848/932. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> As a result, I suggest for kafka-share-groups.sh > >> that > >>>>>> you > >>>>>>>>>> add a > >>>>>>>>>>>>>>>>>>>>>> --members > >>>>>>>>>>>>>>>>>>>>>>>> --verbose option > >>>>>>>>>>>>>>>>>>>>>>>> and only include the new information in the output > >>>>> for > >>>>>>> that > >>>>>>>>>>>>> option, > >>>>>>>>>>>>>>>>>>>>>> rather > >>>>>>>>>>>>>>>>>>>>>>>> than changing the > >>>>>>>>>>>>>>>>>>>>>>>> non-verbose --members output. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> I also make a similar suggestion for > >>>>>>>>>> kafka-consumer-groups.sh > >>>>>>>>>>>>>>>>>>> --members > >>>>>>>>>>>>>>>>>>>>>>>> and only add the > >>>>>>>>>>>>>>>>>>>>>>>> new information for the --verbose output. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> AS2: I strongly suggest that you use MEMBER-EPOCH > >>>>>>> instead of > >>>>>>>>>>>>>>>>>>>>>>>> CONSUMER-EPOCH. > >>>>>>>>>>>>>>>>>>>>>>>> I think it's more confusing not following the > >>>>>>> terminology of > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> KIPs. > >>>>>>>>>>>>>>>>>>>>>> For > >>>>>>>>>>>>>>>>>>>>>>>> one thing, > >>>>>>>>>>>>>>>>>>>>>>>> we already have "member" in the admin client such > >> as > >>>>>>>>>>>>>>>>>>> MemberDescription. > >>>>>>>>>>>>>>>>>>>>>>>> It's not a > >>>>>>>>>>>>>>>>>>>>>>>> new term. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>>>>>> Andrew > >>>>>>>>>>>>>>>>>>>>>>>> ________________________________________ > >>>>>>>>>>>>>>>>>>>>>>>> From: PoAn Yang <pay...@apache.org> > >>>>>>>>>>>>>>>>>>>>>>>> Sent: 25 October 2024 13:55 > >>>>>>>>>>>>>>>>>>>>>>>> To: dev@kafka.apache.org <dev@kafka.apache.org> > >>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [DISCUSS] KIP-1099: Extend > >>>>>>>>>> kafka-consumer-groups > >>>>>>>>>>>>>>>>> command > >>>>>>>>>>>>>>>>>>>>>> line > >>>>>>>>>>>>>>>>>>>>>>>> tool to support new consumer group > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Hi Lucas, > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Thanks for the review! > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> 1) Yes, I add related change for > >>>>> kafka-share-groups.sh > >>>>>> to > >>>>>>>>>> the > >>>>>>>>>>>>> KIP. > >>>>>>>>>>>>>>>>>>> Could > >>>>>>>>>>>>>>>>>>>>>>>> you take a look? Thanks for the suggestion. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> 2) We use CONSUMER-ID as member ID. If we use > >>>>>>> MEMBER-EPOCH > >>>>>>>>>>>>> here, > >>>>>>>>>>>>>>>>>>> users > >>>>>>>>>>>>>>>>>>>>>> may > >>>>>>>>>>>>>>>>>>>>>>>> confuse what is different between CONSUMER and > >>>>> MEMBER. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>>>>>> PoAn > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> On 2024/10/23 13:28:17 Lucas Brutschy wrote: > >>>>>>>>>>>>>>>>>>>>>>>>> Hi Frank, > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> thanks for the KIP! > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> 1) For consistency, should we do the same for > >>>>>>>>>>>>>>>>>>>>>>>>> kafka-share-groups.sh, ShareGroupDescription, > >> etc. ? > >>>>>>> Even > >>>>>>>>>> if > >>>>>>>>>>>>> we do > >>>>>>>>>>>>>>>>>>> not > >>>>>>>>>>>>>>>>>>>>>>>>> implement it right now if the share group > >>>>>> implementation > >>>>>>>>>> may > >>>>>>>>>>>>> still > >>>>>>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>>>>>>>> incomplete, it may make sense to include it in > the > >>>>>> KIP. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> 2) Why call it CONSUMER-EPOCH, not MEMBER-EPOCH? > >>>>> That > >>>>>>> would > >>>>>>>>>>>>> seem > >>>>>>>>>>>>>>>>>>> more > >>>>>>>>>>>>>>>>>>>>>>>>> consistent. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Cheers, > >>>>>>>>>>>>>>>>>>>>>>>>> Lucas > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Oct 23, 2024 at 2:41 PM Frank Yang < > >>>>>>>>>>>>> yangp...@gmail.com> > >>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> I would like to kick off the discussion of > >>>>> KIP-1099. > >>>>>>> This > >>>>>>>>>> KIP > >>>>>>>>>>>>>>>>>>> enhances > >>>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>> kafka-consumer-groups tools to include state > >> which > >>>>> is > >>>>>>>>>>>>> introduced > >>>>>>>>>>>>>>>>> by > >>>>>>>>>>>>>>>>>>>>>>>> KIP-848. > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> KIP-1099: Extend kafka-consumer-groups command > >> line > >>>>>>> tool > >>>>>>>>>> to > >>>>>>>>>>>>>>>>> support > >>>>>>>>>>>>>>>>>>>>>> new > >>>>>>>>>>>>>>>>>>>>>>>>>> consumer group - Apache Kafka - Apache Software > >>>>>>> Foundation > >>>>>>>>>>>>>>>>>>>>>>>>>> < > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> cwiki.apache.org > >>>>>>>>>>>>>>>>>>>>>>>>>> < > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> [image: favicon.ico] > >>>>>>>>>>>>>>>>>>>>>>>>>> < > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> < > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1099%3A+Extend+kafka-consumer-groups+command+line+tool+to+support+new+consumer+group > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> Thank you, > >>>>>>>>>>>>>>>>>>>>>>>>>> PoAn > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >>> > >>> > >> > > >
