Hi, I am also interested in improving the Kafka Streams reset functionality based on KIP-1071. Pausing partitions sounds like a good approach to me, and could be implemented similarly in KIP-848 and KIP-1071. For KS, this would be enough to reset stateless topologies. Stateful topologies need an additional function to reset the local state, but this could be added as a follow-up KIP. Looking forward to reading the KIP!
Cheers, Lucas On Wed, Nov 19, 2025 at 5:22 PM David Jacot <[email protected]> wrote: > > Thanks, Levani! > > On Wed, Nov 19, 2025 at 4:53 PM Levani Kokhreidze <[email protected]> > wrote: > > > Hi David, > > > > Thank you for your reply and sharing your thoughts. > > > > We were already considering that this functionality would be limited to a > > new Consumer Group (CG) protocol. > > > > I also agree that using a separate API for pausing instead of delegating > > everything to the OffsetCommit API seems like a more solid approach. > > > > We will start working on the KIP and initiate the discussion thread > > shortly. > > > > Best > > > > Levani > > > > > > > On 19. Nov 2025, at 14:59, David Jacot <[email protected]> > > wrote: > > > > > > Hi Levani, > > > > > > Thanks for sharing your thoughts! Coincidentally, I was thinking about > > > something similar. I agree that requiring the entire group to shut down > > to > > > reset offsets is an annoying limitation—especially when you only want to > > > reset the offsets of a few partitions that are stuck for whatever reason. > > > With the new consumer rebalance protocol, I think we have an opportunity > > to > > > improve this since the server is now in charge of the assignment. I don’t > > > think we can do much when the classic protocol is used, though. > > > > > > Building on your idea, I would suggest adding the ability to pause > > > partitions within the group. When a partition is paused, the group > > > coordinator would unassign it and keep it aside. We could then relax the > > > OffsetCommit API checks to allow committing offsets for paused > > partitions. > > > This part is actually quite easy to implement. However, we should > > carefully > > > consider the UX (both API and tooling). > > > > > > An alternative design would skip the pause step and delegate everything > > to > > > the OffsetCommit API. I’m not a fan of this approach because it feels > > > error-prone—it becomes too easy to reset offsets by mistake. Requiring an > > > explicit pause makes the process clearer and safer. > > > > > > If you want to proceed, it would be great to draft a KIP for this. If > > > possible, we should also take Streams and KIP-1071 into account. > > > > > > Best, > > > David > > > > > > On Tue, Nov 11, 2025 at 3:23 PM Levani Kokhreidze < > > [email protected]> > > > wrote: > > > > > >> Hi Andrew, > > >> > > >> Thanks for your thoughts! To be clear, at the moment our thinking is to > > >> enable stopping/pausing consumer group from broker side via the Admin > > API. > > >> Which in return makes consumer group inactive. > > >> After that one could use the existing consumer group offset reset > > >> functionality that already exists. > > >> > > >> Hope this makes sense. > > >> > > >> Best, > > >> Levani > > >> > > >>> On 11. Nov 2025, at 16:04, Andrew Mills <[email protected]> wrote: > > >>> > > >>> I could see a situation where you pass a status to Kafka for a specific > > >>> Consumer Group. That status could be resetLatest or resetEarliest and > > >> Kafka > > >>> handles the signal similar to a rebalance but sets the offset based on > > >> the > > >>> earliest/latest. I think anything more, like passing a map of > > partitions > > >>> and offsets, could over complicate things. > > >>> > > >>> -- Andrew > > >>> > > >>> On Tue, Nov 11, 2025, 6:59 AM Levani Kokhreidze < > > [email protected]> > > >>> wrote: > > >>> > > >>>> Hi all, > > >>>> > > >>>> > > >>>> > > >>>> I'd like to get some feedback from the community on the following > > idea. > > >>>> > > >>>> Currently, to safely perform a consumer group offset reset, the > > consumer > > >>>> group must be inactive. In practice, for a production service, this > > >> means > > >>>> the entire service must be stopped to perform the reset. Depending on > > >> the > > >>>> service's criticality, this isn't always possible. > > >>>> > > >>>> The alternative is to build custom tooling for our Kafka clients that > > >>>> allows them to be shut down on request. However, developing this > > >>>> client-side tooling is expensive, and that cost multiplies when you > > >> need to > > >>>> support multiple programming languages. > > >>>> > > >>>> I understand why resetting offsets on an active consumer group is > > >>>> problematic. This leads me to wonder: what is the community's opinion > > on > > >>>> developing more advanced broker-side management for the consumer > > groups? > > >>>> > > >>>> In short, what if we could use the AdminClient to mark a specific > > >> consumer > > >>>> group as "inactive"? This state would then enable us to perform an > > >> offset > > >>>> reset safely, without ever needing to make changes or take actions on > > >> the > > >>>> client side. > > >>>> > > >>>> I appreciate that "marking a consumer group as inactive" is an > > abstract > > >>>> term, and there are many nuances and details to think through. At this > > >>>> point, I'm just looking for super-early feedback from more experienced > > >>>> Kafka developers on whether this general idea makes sense. > > >>>> > > >>>> If it does, my team and I would be very interested in helping to > > deliver > > >>>> this feature. We can start working on a KIP to share with the > > >> community. We > > >>>> would also appreciate any early pointers on how to approach this if > > you > > >>>> think it's a viable idea. > > >>>> > > >>>> > > >>>> > > >>>> Thanks, > > >>>> > > >>>> Levani > > >> > > >> > > > >
