Hi Levani, Thanks for sharing your thoughts! Coincidentally, I was thinking about something similar. I agree that requiring the entire group to shut down to reset offsets is an annoying limitation—especially when you only want to reset the offsets of a few partitions that are stuck for whatever reason. With the new consumer rebalance protocol, I think we have an opportunity to improve this since the server is now in charge of the assignment. I don’t think we can do much when the classic protocol is used, though.
Building on your idea, I would suggest adding the ability to pause partitions within the group. When a partition is paused, the group coordinator would unassign it and keep it aside. We could then relax the OffsetCommit API checks to allow committing offsets for paused partitions. This part is actually quite easy to implement. However, we should carefully consider the UX (both API and tooling). An alternative design would skip the pause step and delegate everything to the OffsetCommit API. I’m not a fan of this approach because it feels error-prone—it becomes too easy to reset offsets by mistake. Requiring an explicit pause makes the process clearer and safer. If you want to proceed, it would be great to draft a KIP for this. If possible, we should also take Streams and KIP-1071 into account. Best, David On Tue, Nov 11, 2025 at 3:23 PM Levani Kokhreidze <[email protected]> wrote: > Hi Andrew, > > Thanks for your thoughts! To be clear, at the moment our thinking is to > enable stopping/pausing consumer group from broker side via the Admin API. > Which in return makes consumer group inactive. > After that one could use the existing consumer group offset reset > functionality that already exists. > > Hope this makes sense. > > Best, > Levani > > > On 11. Nov 2025, at 16:04, Andrew Mills <[email protected]> wrote: > > > > I could see a situation where you pass a status to Kafka for a specific > > Consumer Group. That status could be resetLatest or resetEarliest and > Kafka > > handles the signal similar to a rebalance but sets the offset based on > the > > earliest/latest. I think anything more, like passing a map of partitions > > and offsets, could over complicate things. > > > > -- Andrew > > > > On Tue, Nov 11, 2025, 6:59 AM Levani Kokhreidze <[email protected]> > > wrote: > > > >> Hi all, > >> > >> > >> > >> I'd like to get some feedback from the community on the following idea. > >> > >> Currently, to safely perform a consumer group offset reset, the consumer > >> group must be inactive. In practice, for a production service, this > means > >> the entire service must be stopped to perform the reset. Depending on > the > >> service's criticality, this isn't always possible. > >> > >> The alternative is to build custom tooling for our Kafka clients that > >> allows them to be shut down on request. However, developing this > >> client-side tooling is expensive, and that cost multiplies when you > need to > >> support multiple programming languages. > >> > >> I understand why resetting offsets on an active consumer group is > >> problematic. This leads me to wonder: what is the community's opinion on > >> developing more advanced broker-side management for the consumer groups? > >> > >> In short, what if we could use the AdminClient to mark a specific > consumer > >> group as "inactive"? This state would then enable us to perform an > offset > >> reset safely, without ever needing to make changes or take actions on > the > >> client side. > >> > >> I appreciate that "marking a consumer group as inactive" is an abstract > >> term, and there are many nuances and details to think through. At this > >> point, I'm just looking for super-early feedback from more experienced > >> Kafka developers on whether this general idea makes sense. > >> > >> If it does, my team and I would be very interested in helping to deliver > >> this feature. We can start working on a KIP to share with the > community. We > >> would also appreciate any early pointers on how to approach this if you > >> think it's a viable idea. > >> > >> > >> > >> Thanks, > >> > >> Levani > >
