Hi,  Cooper. Thank you for bringing up the discussion about Kafka 4.0.0.
I tend to release Kafka Connector 4.0 with Flink 2.0.0 and Kafka 3.9.0, or 
Kafka 3.8.1 if If the community is concerned about 3.9.0.

Here are some of my opinions:

1. The plan for Kafka Connector 4.0.0 was proposed[1] a long time ago, mainly 
to support the Flink 2.0 API. Although it has been delayed for some time, 
users' demand for this version is becoming increasingly strong (as we can see 
from emails or Slack), so it is more important to release a version that 
supports Flink 2.0 as soon as possible.

2. For Bump to Kafka 4.0.0, I personally believe that this is a decision that 
requires careful analysis, because rolling back to Kafka client version 3. x 
will require additional workload, and the impact on performance and API 
compatibility may not be reflected solely through IT cases(Of course, I believe 
it will have a positive impact). 
I don't recommend making two major changes in one version,So perhaps 4.1/5.0 is 
more suitable for doing this.

3. As for Kafka 3.9.0, I don't have a big opinion as we have been using Kafka 
client 3.4.0 for almost two years since FLINK-31599[2]. Of course, I also 
approve of using Kafka 3.8.1. 
But for using 3.9.1, the benefits are not very significant and it will block 
the activities of other developers who rely on Flink connector Kafka release 
4.0. As I know, some e2e in Flink rely on Kafka connectors, but they have been 
temporarily removed[3], and development activities[4] for Apache Paimon 
adaptation to Flink 2.0, etc.


Best, Yanquan.

[1]https://lists.apache.org/thread/rl7prqop7wfn2o8j2j9fd96dgr1bjjnx
[2]https://issues.apache.org/jira/browse/FLINK-31599
[3]https://issues.apache.org/jira/browse/FLINK-36413
[4]https://github.com/apache/paimon/issues/5350


> 2025年4月1日 22:50,Tom Cooper <c...@tomcooper.dev> 写道:
> 
> Kafka 4.0.0

Reply via email to