Re: [DISCUSS] KIP-1274: Deprecate and remove support for Classic rebalance protocol in KafkaConsumer

[Matthias J. Sax]

About removing the config `group.protocol` with 6.0. I would advocate to 
do the removal, as originally proposed.

If the config serves not purpose any longer (ie, has only one value), 
there seems to be no reason to keep it (this also help to reduce the 
number of config we have; which is a lot a not easy to navigate for users).

Also, with AK 5.0, we plan to start to log a WARN when the config is 
used (for both values "classic" and "consumer"), and because "consumer" 
will be the default, we would expect users to not specifiy the config at 
all (also not with value "consumer" any longer), but to rely on the 
default. Thus, removing the config with 6.0 would not add any noise for 
all "correctly" setup consumers.

In the end, to avoid WARN logs, with AK 4.3 user would switch from 
"classic" to "consumer", and with AK 5.0 remove the config entirely. At 
least, this is the desired migration path from my understanding.


-Matthias

On 1/28/26 4:49 AM, Lianet Magrans wrote:
Thanks all for the great discussion!

Since all the questions were around the same points I will address them
together:

- I added a section specific to Connect, clarifying that we do want to
upgrade connect usages of the KafkaConsumer, while the Connect-specific
implementation of the classic remains untouched for now and unaffected by
this KIP (to be addressed separately).
- about removing/keeping the group.protocol config after 6.0: agree that it
makes sense to not introduce noise to users that were already defining it
in the right way, so we could keep it and just validate it can only support
the consumer protocol. Updated.
- about the timeline and delaying the start of the deprecation cycle, added
a note on the alternatives with the tradeoffs. One of the main goals is to
encourage adoption in the KafkaConsumer applications, that's why the
proposal of starting the deprecation sooner rather than later (not waiting
on adoption to deprecate). I agree with Matthias' point that as long as we
allow for a safe deprecation cycle (from 4.3 to 5.0 and extended to 6.0),
it shouldn't be too aggressive. Thoughts?

Please take a look and let me know your thoughts. Thanks!

Lianet

On Wed, Jan 28, 2026 at 4:52 AM Andrew Schofield <andrew_schofi...@live.com>
wrote:

HI Lianet et al,
I agree with the proposed schedule and starting people on the long path
to the new protocol seems like the right decision to me.

One comment.

AS1: In phase 3, you are going to remove the `group.protocol` config. That
seems unnecessary to me. If the user has specified
`group.protocol=consumer`
as part of their migration onto the new protocol, they are already doing
the
right thing when we get to AK 6.0 and there’s no reason to remove the
config too. I suggest leaving the config but only permitting the value
“consumer”.

Thanks,
Andrew

On 28 Jan 2026, at 03:42, Matthias J. Sax <mj...@apache.org> wrote:

Thanks for the confirmation on Connect. It does match my understanding.
Maybe I did not express myself clearly enough in my last email. -- I agree
that it would be good to explain explicitly on the KIP.


About "encourage users to use the new `consumer` protocol" vs
"deprecating the `classic` protocol", and don't see much of a difference?
Also, how do we "encourage" them? Based on experience, a deprecation
warning is the best way to encourage them from my POV.

Also, if we want to make "consumer" default with AK 5.0, it seems
reasonable to start the deprecation cycle now. In general, we aim to have a
one year deprecation period, so if we deprecate only in one year, eg 4.6,
we could only change the default if there is also 4.7 and 4.8 release
before 5.0 (or violate the one year guarantee we usually provide). This
sounds unnecessary risky.

If we deprecate with AK 4.3 though, we can make the switch to "consumer"
as default with 5.0, even with much fewer 4.x releases, de-risking the
process.

Given that the goal for 5.0 is not to remove "classic", but only to
change the default, I don't think it's too aggressive. This KIP is somewhat
special as the intended deprecation is not about removal in the next major
release, so I think it's fine.

I would agree that removing "classic" in 5.0 would be too aggressive,
but that's not the proposal.



-Matthias

On 1/27/26 10:23 AM, Ismael Juma wrote:
Hi David,
Thanks for the clarification. For the two points you shared:
1. I think the key point is that Connect handles this via a separate
set of
classes - not the Consumer.
2. Can we make it explicit in the KIP that we will migrate this Connect
usage to the new rebalance protocol by default in 5.0?
In addition, I wonder if it's a good idea to deprecate the classic
consumer
in 4.3 - isn't that a bit aggressive? Perhaps deprecation should wait
until
KIP-848 is more widely used. We can still encourage folks to try the
consumer group protocol. I think a reasonable target for actual
deprecation
would be a 4.x release roughly a year from now.
Ismael
On Tue, Jan 27, 2026 at 7:56 AM David Jacot via dev <
dev@kafka.apache.org>
wrote:
Ismael,

We need to discuss two cases for Connect:

1) Connect uses the classic rebalance protocol to assign its tasks
within a
Connect cluster. This will keep working until we figure out a plan to
migrate this part to another protocol (e.g. Connect protocol following
Streams protocol). This is completely orthogonal from the Consumer.

2) Connect uses Consumers in sources/sinks to read from topics. Those
consumers can use the consumer rebalance protocol without any issues.
They
can just follow the upgrade path and use the new protocol when the
config
changes.

This means that in 6.0, Connect's Consumer will use the new rebalance
protocol whereas Connect's runtime will continue to use the classic
protocol to distribute its tasks.

Lianet, we should perhaps clarify this in the KIP.

Best,
David



On Tue, Jan 27, 2026 at 4:27 PM Ismael Juma <m...@ismaeljuma.com> wrote:

Lianet,

Thanks for the KIP. I think this is a desirable goal, but it seems
premature without a plan for Connect. Removing the config for regular
users
while allowing it for Connect seems problematic. While we figure the
details of that and the deprecation story for the classic rebalance
protocol, we could still agree to a log line during start-up
recommending
`classic` users to switch to the switch to `consumer` (after
appropriate
testing).

Ismael

On Thu, Jan 22, 2026 at 2:05 PM Lianet Magrans <lian...@apache.org>
wrote:

  Hello,

I would like to start the discussion for KIP-1274. This KIP proposes
a
phased plan to drive a smooth evolution of Java client consumer
applications towards the next generation of the rebalance protocol

Here is the KIP:




https://cwiki.apache.org/confluence/display/KAFKA/KIP-1274%3A+Deprecate+and+remove+support+for+Classic+rebalance+protocol+in+KafkaConsumer

Looking forward to hearing from you,

Thanks!
Lianet