Thanks, Levani! On Wed, Nov 19, 2025 at 4:53 PM Levani Kokhreidze <[email protected]> wrote:
> Hi David, > > Thank you for your reply and sharing your thoughts. > > We were already considering that this functionality would be limited to a > new Consumer Group (CG) protocol. > > I also agree that using a separate API for pausing instead of delegating > everything to the OffsetCommit API seems like a more solid approach. > > We will start working on the KIP and initiate the discussion thread > shortly. > > Best > > Levani > > > > On 19. Nov 2025, at 14:59, David Jacot <[email protected]> > wrote: > > > > Hi Levani, > > > > Thanks for sharing your thoughts! Coincidentally, I was thinking about > > something similar. I agree that requiring the entire group to shut down > to > > reset offsets is an annoying limitation—especially when you only want to > > reset the offsets of a few partitions that are stuck for whatever reason. > > With the new consumer rebalance protocol, I think we have an opportunity > to > > improve this since the server is now in charge of the assignment. I don’t > > think we can do much when the classic protocol is used, though. > > > > Building on your idea, I would suggest adding the ability to pause > > partitions within the group. When a partition is paused, the group > > coordinator would unassign it and keep it aside. We could then relax the > > OffsetCommit API checks to allow committing offsets for paused > partitions. > > This part is actually quite easy to implement. However, we should > carefully > > consider the UX (both API and tooling). > > > > An alternative design would skip the pause step and delegate everything > to > > the OffsetCommit API. I’m not a fan of this approach because it feels > > error-prone—it becomes too easy to reset offsets by mistake. Requiring an > > explicit pause makes the process clearer and safer. > > > > If you want to proceed, it would be great to draft a KIP for this. If > > possible, we should also take Streams and KIP-1071 into account. > > > > Best, > > David > > > > On Tue, Nov 11, 2025 at 3:23 PM Levani Kokhreidze < > [email protected]> > > wrote: > > > >> Hi Andrew, > >> > >> Thanks for your thoughts! To be clear, at the moment our thinking is to > >> enable stopping/pausing consumer group from broker side via the Admin > API. > >> Which in return makes consumer group inactive. > >> After that one could use the existing consumer group offset reset > >> functionality that already exists. > >> > >> Hope this makes sense. > >> > >> Best, > >> Levani > >> > >>> On 11. Nov 2025, at 16:04, Andrew Mills <[email protected]> wrote: > >>> > >>> I could see a situation where you pass a status to Kafka for a specific > >>> Consumer Group. That status could be resetLatest or resetEarliest and > >> Kafka > >>> handles the signal similar to a rebalance but sets the offset based on > >> the > >>> earliest/latest. I think anything more, like passing a map of > partitions > >>> and offsets, could over complicate things. > >>> > >>> -- Andrew > >>> > >>> On Tue, Nov 11, 2025, 6:59 AM Levani Kokhreidze < > [email protected]> > >>> wrote: > >>> > >>>> Hi all, > >>>> > >>>> > >>>> > >>>> I'd like to get some feedback from the community on the following > idea. > >>>> > >>>> Currently, to safely perform a consumer group offset reset, the > consumer > >>>> group must be inactive. In practice, for a production service, this > >> means > >>>> the entire service must be stopped to perform the reset. Depending on > >> the > >>>> service's criticality, this isn't always possible. > >>>> > >>>> The alternative is to build custom tooling for our Kafka clients that > >>>> allows them to be shut down on request. However, developing this > >>>> client-side tooling is expensive, and that cost multiplies when you > >> need to > >>>> support multiple programming languages. > >>>> > >>>> I understand why resetting offsets on an active consumer group is > >>>> problematic. This leads me to wonder: what is the community's opinion > on > >>>> developing more advanced broker-side management for the consumer > groups? > >>>> > >>>> In short, what if we could use the AdminClient to mark a specific > >> consumer > >>>> group as "inactive"? This state would then enable us to perform an > >> offset > >>>> reset safely, without ever needing to make changes or take actions on > >> the > >>>> client side. > >>>> > >>>> I appreciate that "marking a consumer group as inactive" is an > abstract > >>>> term, and there are many nuances and details to think through. At this > >>>> point, I'm just looking for super-early feedback from more experienced > >>>> Kafka developers on whether this general idea makes sense. > >>>> > >>>> If it does, my team and I would be very interested in helping to > deliver > >>>> this feature. We can start working on a KIP to share with the > >> community. We > >>>> would also appreciate any early pointers on how to approach this if > you > >>>> think it's a viable idea. > >>>> > >>>> > >>>> > >>>> Thanks, > >>>> > >>>> Levani > >> > >> > >
