Hi Jason, Yes, I meant as a separate KIP. I can start a KIP for that sometime soon.
Thanks. --Vahid From: Jason Gustafson <ja...@confluent.io> To: dev@kafka.apache.org Cc: Kafka Users <us...@kafka.apache.org> Date: 07/21/2017 11:37 AM Subject: Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand > > Regarding your comment about the current limitation on the information > returned for a consumer group, do you think it's worth expanding the API > to return some additional info (e.g. generation id, group leader, ...)? Seems outside the scope of this KIP. Up to you, but I'd probably leave it for future work. -Jason On Thu, Jul 20, 2017 at 4:21 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi Jason, > > Regarding your comment about the current limitation on the information > returned for a consumer group, do you think it's worth expanding the API > to return some additional info (e.g. generation id, group leader, ...)? > > Thanks. > --Vahid > > > > > From: Jason Gustafson <ja...@confluent.io> > To: Kafka Users <us...@kafka.apache.org> > Cc: dev@kafka.apache.org > Date: 07/19/2017 01:46 PM > Subject: Re: [DISCUSS] KIP-175: Additional '--describe' views for > ConsumerGroupCommand > > > > Hey Vahid, > > Thanks for the updates. Looks pretty good. A couple comments: > > 1. For the --state option, should we use the same column-oriented format > as > we use for the other options? I realize there would only be one row, but > the inconsistency is a little vexing. Also, since this tool is working > only > with consumer groups, perhaps we can leave out "protocol type" and use > "assignment strategy" in place of "protocol"? It would be nice to also > include the group generation, but it seems we didn't add that to the > DescribeGroup response. Perhaps we could also include a count of the > number > of members? > 2. It's a little annoying that --subscription and --members share so much > in common. Maybe we could drop --subscription and use a --verbose flag to > control whether or not to include the subscription and perhaps the > assignment as well? Not sure if that's more annoying or less, but maybe a > generic --verbose will be useful in other contexts. > > As for your question on whether we need the --offsets option at all, I > don't have a strong opinion, but it seems to make the command semantics a > little more consistent. > > -Jason > > On Tue, Jul 18, 2017 at 12:56 PM, Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > > > Hi Jason, > > > > I updated the KIP based on your earlier suggestions: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 175%3A+Additional+%27--describe%27+views+for+ConsumerGroupCommand > > The only thing I am wondering at this point is whether it's worth to > have > > a `--describe --offsets` option that behaves exactly like `--describe`. > > > > Thanks. > > --Vahid > > > > > > > > From: "Vahid S Hashemian" <vahidhashem...@us.ibm.com> > > To: dev@kafka.apache.org > > Cc: Kafka Users <us...@kafka.apache.org> > > Date: 07/17/2017 03:24 PM > > Subject: Re: [DISCUSS] KIP-175: Additional '--describe' views for > > ConsumerGroupCommand > > > > > > > > Hi Jason, > > > > Thanks for your quick feedback. Your suggestions seem reasonable. > > I'll start updating the KIP accordingly and will send out another note > > when it's ready. > > > > Regards. > > --Vahid > > > > > > > > > > From: Jason Gustafson <ja...@confluent.io> > > To: dev@kafka.apache.org > > Cc: Kafka Users <us...@kafka.apache.org> > > Date: 07/17/2017 02:11 PM > > Subject: Re: [DISCUSS] KIP-175: Additional '--describe' views for > > ConsumerGroupCommand > > > > > > > > Hey Vahid, > > > > Hmm... If possible, it would be nice to avoid cluttering the default > > option > > too much, especially if it is information which is going to be the same > > for > > all members (such as the generation). My preference would be to use the > > --state option that you've suggested for that info so that we can > > represent > > it more concisely. > > > > The reason I prefer the current output is that it is clear every entry > > corresponds to a partition for which we have committed offset. Entries > > like > > this look strange: > > > > TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET > > LAG CONSUMER-ID > > HOST CLIENT-ID > > - - - - > > - consumer4-e173f09d-c761-4f4e-95c7-6fb73bb8fbff > > /127.0.0.1 > > consumer4 > > - - - - > > - consumer5-7b80e428-f8ff-43f3-8360-afd1c8ba43ea > > /127.0.0.1 > > consumer5 > > > > It makes me think that the consumers have committed offsets for an > unknown > > partition. The --members option seems like a clearer way to communicate > > the > > fact that there are some members with no assigned partitions. > > > > A few additional suggestions: > > > > 1. Maybe we can rename --partitions to --offsets or --committed-offsets > > and > > the output could match the default output (in other words, --offsets is > > treated as the default switch). Seems no harm including the assignment > > information if we have it. > > 2. Along the lines of Onur's comment, it would be nice if the --members > > option included the list of assignment strategies that the consumer > joined > > with (round-robin, range, etc). This list should always be small. > > 3. Thinking a little more, I'm not sure how necessary a --topics option > > is. > > The --partitions (or --offsets) option already shows the current > > assignment. Maybe --topics could be --subscription and just list the > > topics > > that the members subscribed to? > > > > Thanks, > > Jason > > > > On Mon, Jul 17, 2017 at 11:04 AM, Vahid S Hashemian < > > vahidhashem...@us.ibm.com> wrote: > > > > > Jason, Onur, thank you for reviewing the KIP. > > > > > > Regarding the default `--describe` option, so far there have been a > few > > > suggestions that conflict a bit. Here are the suggestions: > > > - Keep the current behavior exactly as is (Edo, Jeff) > > > - Remove members with no assignments from the current result set > (Jason) > > > - Add additional status info to the result set (Onur) -- I assume the > > > additional status (which are group related info, rather than group > > member > > > related) will appear in the result separate from the member table > (e.g., > > > before the table) > > > > > > One thing we could do to remain as close as possible to these > > suggestions > > > is trim the resulting rows as per Jason's suggestion, and add the > > > additional details that Onur suggested. Would this work for everyone? > > Edo, > > > Jeff, what do you think? > > > If so, I'll update the KIP accordingly. > > > > > > Some of the other updates based on the feedback received: > > > * "--describe --members" will not include a topic(partitions) column. > > > Instead there will be a #Partitions (number of partitions assigned to > > this > > > member) column > > > * "--describe --topics" will be added to list topic partitions in the > > > group and the relevant info > > > * "--describe --state" will be added to report group related info, > such > > as > > > state, protocol, ... > > > > > > Thanks. > > > --Vahid > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
