Hi David, Thank you for your reply and sharing your thoughts.
We were already considering that this functionality would be limited to a new Consumer Group (CG) protocol. I also agree that using a separate API for pausing instead of delegating everything to the OffsetCommit API seems like a more solid approach. We will start working on the KIP and initiate the discussion thread shortly. Best Levani > On 19. Nov 2025, at 14:59, David Jacot <[email protected]> wrote: > > Hi Levani, > > Thanks for sharing your thoughts! Coincidentally, I was thinking about > something similar. I agree that requiring the entire group to shut down to > reset offsets is an annoying limitation—especially when you only want to > reset the offsets of a few partitions that are stuck for whatever reason. > With the new consumer rebalance protocol, I think we have an opportunity to > improve this since the server is now in charge of the assignment. I don’t > think we can do much when the classic protocol is used, though. > > Building on your idea, I would suggest adding the ability to pause > partitions within the group. When a partition is paused, the group > coordinator would unassign it and keep it aside. We could then relax the > OffsetCommit API checks to allow committing offsets for paused partitions. > This part is actually quite easy to implement. However, we should carefully > consider the UX (both API and tooling). > > An alternative design would skip the pause step and delegate everything to > the OffsetCommit API. I’m not a fan of this approach because it feels > error-prone—it becomes too easy to reset offsets by mistake. Requiring an > explicit pause makes the process clearer and safer. > > If you want to proceed, it would be great to draft a KIP for this. If > possible, we should also take Streams and KIP-1071 into account. > > Best, > David > > On Tue, Nov 11, 2025 at 3:23 PM Levani Kokhreidze <[email protected]> > wrote: > >> Hi Andrew, >> >> Thanks for your thoughts! To be clear, at the moment our thinking is to >> enable stopping/pausing consumer group from broker side via the Admin API. >> Which in return makes consumer group inactive. >> After that one could use the existing consumer group offset reset >> functionality that already exists. >> >> Hope this makes sense. >> >> Best, >> Levani >> >>> On 11. Nov 2025, at 16:04, Andrew Mills <[email protected]> wrote: >>> >>> I could see a situation where you pass a status to Kafka for a specific >>> Consumer Group. That status could be resetLatest or resetEarliest and >> Kafka >>> handles the signal similar to a rebalance but sets the offset based on >> the >>> earliest/latest. I think anything more, like passing a map of partitions >>> and offsets, could over complicate things. >>> >>> -- Andrew >>> >>> On Tue, Nov 11, 2025, 6:59 AM Levani Kokhreidze <[email protected]> >>> wrote: >>> >>>> Hi all, >>>> >>>> >>>> >>>> I'd like to get some feedback from the community on the following idea. >>>> >>>> Currently, to safely perform a consumer group offset reset, the consumer >>>> group must be inactive. In practice, for a production service, this >> means >>>> the entire service must be stopped to perform the reset. Depending on >> the >>>> service's criticality, this isn't always possible. >>>> >>>> The alternative is to build custom tooling for our Kafka clients that >>>> allows them to be shut down on request. However, developing this >>>> client-side tooling is expensive, and that cost multiplies when you >> need to >>>> support multiple programming languages. >>>> >>>> I understand why resetting offsets on an active consumer group is >>>> problematic. This leads me to wonder: what is the community's opinion on >>>> developing more advanced broker-side management for the consumer groups? >>>> >>>> In short, what if we could use the AdminClient to mark a specific >> consumer >>>> group as "inactive"? This state would then enable us to perform an >> offset >>>> reset safely, without ever needing to make changes or take actions on >> the >>>> client side. >>>> >>>> I appreciate that "marking a consumer group as inactive" is an abstract >>>> term, and there are many nuances and details to think through. At this >>>> point, I'm just looking for super-early feedback from more experienced >>>> Kafka developers on whether this general idea makes sense. >>>> >>>> If it does, my team and I would be very interested in helping to deliver >>>> this feature. We can start working on a KIP to share with the >> community. We >>>> would also appreciate any early pointers on how to approach this if you >>>> think it's a viable idea. >>>> >>>> >>>> >>>> Thanks, >>>> >>>> Levani >> >>
