Hi Andrew, Thanks for the KIP. I have one question. AM1: Since this KIP adds --by-duration to kafka-share-groups.sh while share groups also accept share.auto.offset.reset=by_duration:… as a group config. I can find share.auto.offset.reset=by_duration:PT3H (config, kafka-configs.sh): an init-time policy the broker applies only when no SPSO exists. I can set the config while there are active members but the config update is not honoured. It's only honoured when a new group is specified. Can you please clarify the expected behaviour when using `kafka-share-groups.sh --by-duration` reset option?
Regards, Apoorv Mittal On Thu, Aug 13, 2026 at 1:21 PM Andrew Schofield <[email protected]> wrote: > Hi Aditya, > Thanks for your reply. I'm not dismissing your concern, but I'm not > entirely happy with a way forward yet. > > AK1: I wonder whether we could use a pipe here to avoid the intermediate > file. After all, export uses stdout as the destination, so if we could use > stdin as the source. For example, I've used '-' with --from-file below to > indicate stdin. It could be --from-stdin or similar too. > > $ bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 > --export-offsets --group CG1 --topic T1 --to-current | > bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 > --reset-offsets --group CG2 --from-file - --topic T1 --execute > > This would still allow resetting between different group types, which is > actually the use case that got me into this, taking offsets from a consumer > group to initialise a share group. > > Thanks, > Andrew > > On 2026/08/12 18:12:08 Aditya Kousik wrote: > > Hi Andrew. > > > > Thanks for addressing AK2-4: lgtm > > > > On AK1: I’m thinking from a user experience standpoint. The teams > managing the kafka clusters and the ones using the clients diverge - but > the Admin role may be exposed via self-service typically via API requests > and a K8S CRD. I’m thinking of the latter case where Kafka SRE allow a thin > wrapper for client teams to manage their own groups and topics and also > isolating the environment where the scripts can be run. In such a case, it > becomes cumbersome to fetch/write the csv file from a Pod that runs the > migration ; it may not have any volume to persist it or running kubectl to > scp the file is usually denied. So in such a case, the migrate offsets > script could just be a single command that fetches offsets from one group > and applies it to another (or several) without the middleman CSV. > > > > Thanks, > > Aditya > > > > > On Aug 12, 2026, at 06:39, Andrew Schofield <[email protected]> > wrote: > > > > > > Hi Aditya, > > > Thanks for your reply. Sorry for the delay. > > > > > > AK1: One of the advantages of the file is that it's possible to export > from one group type and import into another. You can also export offsets > for multiple groups, and this changes the file format (4 columns instead of > 3). Sadly, this is all existing behaviour that I only learnt by trial and > error. Personally, I think that export-import is a relatively infrequent > operation, and the KIP makes enough of a step forward. I wonder if you > could elaborate a bit more on your idea. Feel free to push back if you > disagree. > > > > > > AK2: The KIP does not mention this and I wasn't quite sure where to > add it. You are correct. Should we improve the javadoc for the admin client? > > > > > > AK3: Yes, Admin is better. KIP updated. > > > > > > AK4: The DescribeGroupsGeneric RPC is intended as a way of getting a > minimal set of group information for groups of any group types. It is used > in Admin.describeGroups and this would be the initial Admin call from > bin/kafka-XXXX-groups.sh when exporting or resetting offsets. It gives a > way to check the group type and existence unambiguously. By contrast, if > you use Admin.describeXXXGroups() for an existing group ID of the wrong > type, the error code is GROUP_ID_NOT_FOUND. > > > > > > AK4.1: I didn't include the members intentionally. This RPC is > agnostic to group type. I think the common subset of group attributes is > group ID, group type, group state and coordinator. If you dig deeper, you > find that the group types differ. For example, assignments in consumer > groups and share groups are partitions, while in streams groups they are > tasks. I expect we'll add connect groups one day, and until we have the > KIP, I wouldn't want to assume the details of membership. > > > > > > AK4.2: This is following existing convention I'm afraid and I want to > be consistent. If the broker does not send back a list of ACL operations, > then this will be null. This is different than an empty set which means > "you are not authorized", as opposed to "you don't know anything about > authorized operations". > > > > > > Thanks, > > > Andrew > > > > > >> On 2026/07/27 05:45:27 Aditya Kousik wrote: > > >> A small correction. > > >> > > >> I meant to add a statement before the numbered comments. > > >> > > >> “Since the KIP motivation revolves around usability of the script, > would you be willing to consider the following:” > > >> > > >> Also, typo on AK2. “…As long as there aren’t any active members in > the group subscription”. > > >> > > >> -Aditya > > >> > > >>>> On Jul 25, 2026, at 23:53, Aditya Kousik <[email protected]> > wrote: > > >>> > > >>> Hi Andrew, > > >>> > > >>> Thanks for this KIP, I’ve personally faced more than one scenario > where I’ve had to use the admin client to update the offsets. > > >>> > > >>> The main uses cases teams face ( and TIL about the various options > the script offers): > > >>> Teams migrating to a new consumer group from an existing one but > don’t want to commit to either (or duration for newer clients) auto offset > reset policy on the new group. > > >>> Teams want to seek past certain offsets in partitions because they > have poison pill messages/corrupted deserialization story and they have no > control over the data. In this case, they actually want latest offset > strategy but most don’t even realise that it applies to streams out of > range and are surprised/frustrated that the property doesn’t move their > position in the queues. (I see we can do this already in the script today) > > >>> When I migrated our clients from 2.x directly to 4.x, out of > abundance of caution, I wrote an hourly cron job that dumps the snapshot of > group_id,topic,partition,offset into csv files so that if I had to revert a > few affected partitions I can trim the csv down to those partitions and > alter them with the admin client. > > >>> > > >>> AK1. A single command line with args —source-group-id > —target-group-id —export so that the intermediate CSV file can be skipped. > > >>> AK2. Minor: The KIP does not mention the alter/list consumer group > offsets Admin call. alter only works as long as there are active members in > the group subscription. I was bitten by this in the past; the documentation > was sparse around that expectation but it does make sense. > > >>> AK3. KIP references AdminClient but I believe this is deprecated in > favour of just Admin which is the interface that has the methods? > > >>> AK4. How do we leverage the describe groups RPC when using the > script/admin client? I’m trying to imagine flows other than the export > group offsets. > > >>> AK4.1 Can we list active members/offset state in a given group in > addition to the GroupState? This has helped identify zombie processes to > hunt down. The broker’s view would be useful here. (related to AK2) > > >>> AK4.2 Could we return empty set of acl operations instead of null? > > >>> > > >>> Best, > > >>> Aditya > > >>> > > >>> > > >>>> On 2026/06/17 06:34:11 Andrew Schofield wrote: > > >>>> Hi, > > >>>> I’d like to start discussion on a small KIP to improve the > usability of the command-line tools for resetting group offsets. When > implementing KIP-1323 and reviewing the PRs, it became clear that the > usability of this area of Kafka isn’t great. This KIP improves the > usability of these tools. > > >>>> > > >>>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1359%3A+Improve+usability+of+resetting+group+offsets > > >>>> > > >>>> Thanks, > > >>>> Andrew > > >>>> > > >> > > >
