Hi Tom, Fabian, Thanks for the updates on the v4.0.1 release. I noticed that v4.0.1 is going out with Flink 2.0 (link <https://github.com/apache/flink-connector-kafka/blob/v4.0.1-rc2/pom.xml#L56>). Do you have a timeline or target window in mind for the Flink Connector Kafka 5.0 release that will include Flink 2.1 support?
Thanks, Weiqing On Mon, Aug 11, 2025 at 7:41 AM Tom Cooper <c...@tomcooper.dev> wrote: > Hi Fabian, > > Sorry, for the late reply, this message somehow ended up in my spam > filter!? > > I think having the Flink 2.1 upgrade included in the move to Flink > Connector Kafka 5.0 makes sense. > I am hoping to find the time to work on the upgrade to Flink 2.1 at end of > this week or next. > Unless, of course, you are plan to work on that? > > Regards, > > Tom Cooper > @tomcooper.dev | https://tomcooper.dev > > > On Tuesday, 29 July 2025 at 09:08, Fabian Paul <fp...@confluent.io.INVALID> > wrote: > > > Hi Tom, > > > > Sounds good to me, I can start with the 4.0.1 release. > > Regarding the 5.0 release, I am not super sure yet what to include. > > Since releasing always takes some effort, I would also be okay with > > doing the 5.0 release with incorporating Flink 2.1. The connector > > already offers a release that is compatible with Flink 2.0, and in > > theory, 2.1 should not introduce breaking changes that affect the > > connector. > > > > Best, > > Fabian > > > > On Mon, Jul 28, 2025 at 11:03 AM Tom Cooper c...@tomcooper.dev wrote: > > > > > Hi Fabian, > > > > > > You make a good point, as there are only dependency updates, a 4.0.1 > release makes more sense. > > > > > > At this point the 5.0 connector release could include the soon to be > released Kafka 4.0.1 client libraries (the RC for that is out already). > > > I assume we would want to leave the flink 2.1 upgrade to a future 5.1 > release? > > > > > > Thanks for looking at this. > > > > > > Regards, > > > > > > Tom Cooper > > > @tomcooper.dev | https://tomcooper.dev > > > > > > On Monday, 28 July 2025 at 09:51, Fabian Paul fp...@apache.org wrote: > > > > > > > Hi Tom, > > > > > > > > Thanks for starting this discussion. I think it's a good idea to do > > > > another 4.1.0 release before proceeding with 5.0 to offer a release > > > > with the vulnerability fixed without requiring users to upgrade to > > > > Kafka 4.0. Is there a reason you prefer to do the 4.1.0 release > > > > instead of the 4.0.1 release? I reviewed the changes between the > > > > current main and the release 4.0.0 [1], and they are mostly > dependency > > > > upgrades and some fixes, but without any new features. What do you > > > > think about doing a 4.0.1 release and then kicking off 5.0.0 with the > > > > Kafka client upgrade? > > > > > > > > Best, > > > > Fabian > > > > > > > > [1] > https://github.com/apache/flink-connector-kafka/compare/v4.0...main > > > > > > > > On Fri, Jul 25, 2025 at 11:58 AM Tom Cooper c...@tomcooper.dev > wrote: > > > > > > > > > Bumping this thread as we are now ready to merge the Kafka 4.0.0 > client update PR [1]. This will bump the major version of the connector to > 5.0, as we are dropping support for Kafka brokers running version 2.0.0 or > earlier. > > > > > > > > > > However, I still think it would be worth doing a 4.1.0 release of > the connector (with the Kafka 3.9.1 client), before the Kafka 4.0.0 client > update is merged. > > > > > > > > > > The current Flink Kafka Connector (4.0) has a critical CVE [2], > which is patched in the 3.9.1 Kafka client library (which the current main > branch of the Flink connector is using). Doing a 4.1 release of the > connector would cover any users of older Kafka versions that want this CVE > patched and also give a stable release of the connector using a point > release of the Kafka client (with all the bug fixes that entails). This > would be a good option for users who don't want to jump straight onto the > new major Kafka client version. > > > > > > > > > > What do people think? > > > > > > > > > > Tom Cooper > > > > > @tomcooper.dev | https://tomcooper.dev > > > > > > > > > > [1] https://github.com/apache/flink-connector-kafka/pull/161 > > > > > [2] https://nvd.nist.gov/vuln/detail/CVE-2025-27817 > > > > > > > > > > On Wednesday, 9 July 2025 at 09:35, Tom Cooper c...@tomcooper.dev > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I would like to start a conversation about releases for the > Flink Connector Kafka project. > > > > > > > > > > > > We have recently updated [0] to version 3.9.1 of the Kafka > client library, which fixes a critical CVE [1]. With that in mind, I think > it would be prudent to have a 4.1.0 release as soon as possible that > includes this. It would also be good to include the dependency bumps from > this PR [2] in that release. > > > > > > > > > > > > With the 4.1.0 release out, we could then move to looking at the > Kafka 4.0 upgrade (there is already a PR [3] for that). The main point with > the Kafka 4.0 upgrade is that it drops support for Kafka brokers running > version 2.0.0 and lower. Given this, I think it would make sense to move > the Connector version to 5.0.0 and maybe even move to Flink 2.1.0 (which > should be available in a month or so). This 5.0.0 release could also remove > all the Zookeeper specific test infra and move to KRaft based clusters for > testing. We could also move to a new, updated Flink Connector Parent pom > version [4] which would harmonise the java versions and plugins with the > main Flink project. > > > > > > > > > > > > I think, if the above is acceptable, that these changes warrant > a major version bump. Users of older Kafka clusters would still be able to > use 4.1.0 (which is an argument for making sure that release has the most > up-to-date dependencies). > > > > > > > > > > > > Anyway, I would love to hear what the community think of the > above. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Tom Cooper > > > > > > @tomcooper.dev | https://tomcooper.dev > > > > > > > > > > > > [0] https://github.com/apache/flink-connector-kafka/pull/180 > > > > > > [1] https://nvd.nist.gov/vuln/detail/CVE-2025-27817 > > > > > > [2] https://github.com/apache/flink-connector-kafka/pull/181 > > > > > > [3] https://github.com/apache/flink-connector-kafka/pull/161 > > > > > > [4] > https://github.com/apache/flink-connector-shared-utils/pull/48 >
