Hi Mickael, done.
Cheers, Lucas On Thu, Jul 17, 2025 at 2:18 PM Mickael Maison <mickael.mai...@gmail.com> wrote: > > Hi Lucas, > > Ok, let me know once you're done. > > Thanks, > Mickael > > On Thu, Jul 17, 2025 at 11:08 AM Lucas Brutschy > <lbruts...@confluent.io.invalid> wrote: > > > > Hi Mickael, > > > > we have been seeing a race condition happening in the initialization > > of the state updater. We'd like to revert 4d6cf3efef > > which changed the initialization of the state updater and is anyway > > linked to KAFKA-18913 that was moved to 4.2. > > > > Let me know if you accept the change. > > > > Cheers, > > Lucas > > > > On Wed, Jul 16, 2025 at 6:47 PM Paolo Patierno <paolo.patie...@gmail.com> > > wrote: > > > > > > Hi Jun, > > > oh yes sorry!! It's NOW fixed :-D Thanks! > > > > > > On Wed, 16 Jul 2025 at 18:38, Jun Rao <j...@confluent.io.invalid> wrote: > > > > > > > Hi, Paolo, > > > > > > > > Thanks for the reply. I guess that you mean the metadata version issue > > > > is > > > > NOW fixed, instead of NOT fixed? > > > > > > > > Jun > > > > > > > > On Wed, Jul 16, 2025 at 8:57 AM Paolo Patierno > > > > <paolo.patie...@gmail.com> > > > > wrote: > > > > > > > > > Hi, > > > > > I already did on the PR but let me confirm here as well that the > > > > > metadata > > > > > version issues is not fixed, I was able to test it locally within our > > > > > Strimzi unit tests. > > > > > > > > > > Thanks, > > > > > Paolo > > > > > > > > > > Twitter : @ppatierno <http://twitter.com/ppatierno> > > > > > Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno> > > > > > GitHub : ppatierno <https://github.com/ppatierno> > > > > > > > > > > On Wed, 16 Jul 2025, 17:43 Mickael Maison, <mickael.mai...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hi Chia-Ping, > > > > > > > > > > > > Sure, thanks > > > > > > > > > > > > Mickael > > > > > > > > > > > > On Wed, Jul 16, 2025 at 4:07 PM Chia-Ping Tsai <chia7...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > > hi Mickael > > > > > > > > > > > > > > I'm going to merge the patch for > > > > > > https://issues.apache.org/jira/browse/KAFKA-19427. The bug could > > > > > > lead > > > > to > > > > > > an OOM in 4.x. The risk is low since the patch is small. Hence, I > > > > > > plan > > > > to > > > > > > backport it to 4.1.0 if you agree to accept it. > > > > > > > > > > > > > > Best, > > > > > > > Chia-Ping > > > > > > > > > > > > > > On 2025/07/15 08:43:40 Mickael Maison wrote: > > > > > > > > Hi Bill, > > > > > > > > > > > > > > > > Yes you can go ahead and backport that to 4.1. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Mickael > > > > > > > > > > > > > > > > On Mon, Jul 14, 2025 at 8:42 PM Bill Bejeck <bbej...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > > > > Hi Mickael, > > > > > > > > > > > > > > > > > > I've found another blocker as well KAFKA-19504 > > > > > > > > > <https://issues.apache.org/jira/browse/KAFKA-19504>, and we > > > > have a > > > > > > PR ( > > > > > > > > > https://github.com/apache/kafka/pull/20166). > > > > > > > > > While this is not a blocker in the strict sense, it's severe > > > > enough > > > > > > that we > > > > > > > > > shouldn't let this ship as is. It's minor change with very > > > > > > > > > low > > > > > risk. > > > > > > > > > > > > > > > > > > On Mon, Jul 14, 2025 at 2:09 PM Jun Rao > > > > > > > > > <j...@confluent.io.invalid > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi, Paolo, > > > > > > > > > > > > > > > > > > > > Thanks for identifying the issue. We now have a PR ( > > > > > > > > > > https://github.com/apache/kafka/pull/20137) to bump up the > > > > > latest > > > > > > > > > > production metadata version. > > > > > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > > > On Mon, Jul 14, 2025 at 1:58 AM Paolo Patierno < > > > > > > paolo.patie...@gmail.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > I think I found something which could be a blocker for the > > > > > > release. > > > > > > > > > > > I was adding the support for Apache Kafka 4.1.0 to Strimzi > > > > and > > > > > > running > > > > > > > > > > our > > > > > > > > > > > tests but some of them are failing with the following > > > > > > > > > > > error. > > > > > > > > > > > > > > > > > > > > > > java.lang.IllegalArgumentException: Version 4.1 is not a > > > > valid > > > > > > version. > > > > > > > > > > The > > > > > > > > > > > minimum version is 3.3-IV3 and the maximum version is > > > > > > > > > > > 4.2-IV1 > > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.kafka.server.common.MetadataVersion.lambda$fromVersionString$0(MetadataVersion.java:356) > > > > > > > > > > > at > > > > > > java.base/java.util.Optional.orElseThrow(Optional.java:403) > > > > > > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.kafka.server.common.MetadataVersion.fromVersionString(MetadataVersion.java:354) > > > > > > > > > > > > > > > > > > > > > > In these tests we use the MetadataVersion class to verify > > > > > > > > > > > the > > > > > > validity of > > > > > > > > > > > metadata but it fails when passing something like "4.1" > > > > (while > > > > > > they work > > > > > > > > > > > just fine with all previous releases, i.e. "4.0", "3.9" > > > > > > > > > > > and > > > > so > > > > > > on). > > > > > > > > > > > > > > > > > > > > > > The exception, thrown by the fromVersionString method, > > > > > > > > > > > seems > > > > to > > > > > > be > > > > > > > > > > related > > > > > > > > > > > to "4.1" missing as key entry within the IBP_VERSIONS map > > > > > > > > > > > and > > > > > > such entry > > > > > > > > > > > seems to be missing because, if you go through the static > > > > > > initialization > > > > > > > > > > of > > > > > > > > > > > the IBP_VERSIONS map, the "4.1" entry is not added due to > > > > > > > > > > > the > > > > > > > > > > > metadataVersion.isProduction() which results false for > > > > > > > > > > > "4.1" > > > > > > because the > > > > > > > > > > > LATEST_PRODUCTION constant is defined as IBP_4_0_IV3 > > > > > > > > > > > while I > > > > > > would assume > > > > > > > > > > > should be something like IBP_4_1_IV1 instead. > > > > > > > > > > > > > > > > > > > > > > I think this issue wasn't caught during Kafka upstream > > > > testing > > > > > > because > > > > > > > > > > the > > > > > > > > > > > MetadataVersionTest.testFromVersionString is missing the > > > > > > > > > > > test > > > > > > for "4.1" > > > > > > > > > > (as > > > > > > > > > > > we have for all other versions). So it's missing something > > > > like > > > > > > > > > > > > > > > > > > > > > > assertEquals(IBP_4_1_IV1, > > > > > > MetadataVersion.fromVersionString("4.1")); > > > > > > > > > > > assertEquals(IBP_4_1_IV0, > > > > > > > > > > > MetadataVersion.fromVersionString("4.1-IV0")); > > > > > > > > > > > assertEquals(IBP_4_1_IV1, > > > > > > > > > > > MetadataVersion.fromVersionString("4.1-IV1")); > > > > > > > > > > > > > > > > > > > > > > Let's add that this issue seems to break the storage tool > > > > where > > > > > > you can > > > > > > > > > > > pass the --release-version, but if you run it by passing > > > > "4.1" > > > > > > you get > > > > > > > > > > > exactly the same exception. > > > > > > > > > > > > > > > > > > > > > > > bin/kafka-storage.sh format --standalone -t > > > > > > kEzc4vk3TIKhCQKsh40klQ -c > > > > > > > > > > > config/server.properties --release-version 4.1 > > > > > > > > > > > Exception in thread "main" > > > > java.lang.IllegalArgumentException: > > > > > > Version > > > > > > > > > > 4.1 > > > > > > > > > > > is not a valid version. The minimum version is 3.3-IV3 and > > > > the > > > > > > maximum > > > > > > > > > > > version is 4.2-IV1 > > > > > > > > > > > > > > > > > > > > > > Finally, I would add that the error message itself is > > > > > misleading > > > > > > as it > > > > > > > > > > > mentions 4.2-IV1 as maximum version which would not exist > > > > > > > > > > > at > > > > > all > > > > > > if you > > > > > > > > > > are > > > > > > > > > > > dealing with a 4.1 release. This misleading message > > > > > > > > > > > should be > > > > > > related to > > > > > > > > > > > MetadataVersion.latestTesting() method which is getting > > > > > > > > > > > last > > > > > > value from > > > > > > > > > > > VERSIONS which is filled with all MetadataVersion enums > > > > > > including the 4.2 > > > > > > > > > > > placeholders (which AFAIU is actually a new practice). > > > > > > > > > > > All of > > > > > > that has > > > > > > > > > > also > > > > > > > > > > > an impact on the ClusterTest annotation class as well as > > > > > > > > > > FeatureCommandTest > > > > > > > > > > > messages to be assert. So that 4.2 seems to be there on > > > > purpose > > > > > > but not > > > > > > > > > > > sure why. > > > > > > > > > > > > > > > > > > > > > > I would be happy to contribute about the first issue which > > > > > > raises the > > > > > > > > > > > exception. Not sure about the misleading error message > > > > because > > > > > > of the > > > > > > > > > > usage > > > > > > > > > > > of the 4.2 placeholders within the VERSIONS collection. > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > Paolo Patierno > > > > > > > > > > > > > > > > > > > > > > ---------- Forwarded message --------- > > > > > > > > > > > From: Mickael Maison <mickael.mai...@gmail.com> > > > > > > > > > > > Date: Fri, Jul 11, 2025 at 11:53 PM > > > > > > > > > > > Subject: [VOTE] 4.1.0 RC0 > > > > > > > > > > > To: dev <dev@kafka.apache.org>, Users < > > > > us...@kafka.apache.org > > > > > >, > > > > > > > > > > > kafka-clients <kafka-clie...@googlegroups.com> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hello Kafka users, developers and client-developers, > > > > > > > > > > > > > > > > > > > > > > This is the first candidate for release of Apache Kafka > > > > 4.1.0. > > > > > > > > > > > > > > > > > > > > > > The release brings many new features and improvements: > > > > > > > > > > > - Mechanism for plugin to register metrics > > > > > > > > > > > - Allow running multiple versions of connector plugins > > > > > > > > > > > - Queue are now in preview access > > > > > > > > > > > - New Streams rebalance protocol in early access > > > > > > > > > > > - Support for OAuth jwt-bearer grant type > > > > > > > > > > > - Improved metadata replication > > > > > > > > > > > > > > > > > > > > > > Release notes for the 4.1.0 release: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/RELEASE_NOTES.html > > > > > > > > > > > > > > > > > > > > > > *** Please download, test and vote by Monday July 21 > > > > > > > > > > > > > > > > > > > > > > Kafka's KEYS file containing PGP keys we use to sign the > > > > > release: > > > > > > > > > > > https://kafka.apache.org/KEYS > > > > > > > > > > > > > > > > > > > > > > * Release artifacts to be voted upon (source and binary): > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/ > > > > > > > > > > > > > > > > > > > > > > * Docker release artifacts to be voted upon: > > > > > > > > > > > apache/kafka:4.1.0-rc0 > > > > > > > > > > > apache/kafka-native:4.1.0-rc0 > > > > > > > > > > > > > > > > > > > > > > * Maven artifacts to be voted upon: > > > > > > > > > > > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > > > > > > > > > > > > > > > > > * Javadoc: > > > > > > > > > > > > > > > > https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc0/javadoc/ > > > > > > > > > > > > > > > > > > > > > > * Tag to be voted upon (off 4.1 branch) is the 4.1.0 tag: > > > > > > > > > > > https://github.com/apache/kafka/releases/tag/4.1.0-rc0 > > > > > > > > > > > > > > > > > > > > > > * Documentation: > > > > > > > > > > > https://kafka.apache.org/41/documentation.html > > > > > > > > > > > PR (https://github.com/apache/kafka-site/pull/702) is not > > > > > > merged yet > > > > > > > > > > > > > > > > > > > > > > * Protocol: > > > > > > > > > > > https://kafka.apache.org/41/protocol.html > > > > > > > > > > > PR (https://github.com/apache/kafka-site/pull/702) is not > > > > > > merged yet > > > > > > > > > > > > > > > > > > > > > > * Successful CI builds for the 4.1 branch: > > > > > > > > > > > Unit/integration tests: > > > > > > > > > > > https://github.com/apache/kafka/actions/runs/16198079614 > > > > > > > > > > > * System tests results: > > > > > > > > > > > - core: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://drive.google.com/file/d/1JLpFBFbPV1N6cdh9EA9TGw9S5SZwkIpv/view?usp=drive_link > > > > > > > > > > > - non-core: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://drive.google.com/file/d/15jCnTS-dssSHauW7uWxroiupkWaeUK7A/view?usp=drive_link > > > > > > > > > > > I got 2 failures in the system tests: > > > > > > > > > > > - > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 'tests/kafkatest/tests/client/consumer_test.py::OffsetValidationTest.test_broker_rolling_bounce@ > > > > > > > > > > > > > > > > {"metadata_quorum":"ISOLATED_KRAFT","group_protocol":"classic"}' > > > > > > > > > > > - > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 'tests/kafkatest/tests/core/network_degrade_test.py::NetworkDegradeTest.test_rate@ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > {"task_name":"rate-1000-latency-50","device_name":"eth0","latency_ms":50,"rate_limit_kbit":1000000,"metadata_quorum":"COMBINED_KRAFT"}' > > > > > > > > > > > > > > > > > > > > > > * Successful Docker Image Github Actions Pipeline for 4.1 > > > > > branch: > > > > > > > > > > > Docker Build Test Pipeline (JVM): > > > > > > > > > > > https://github.com/apache/kafka/actions/runs/16172687407/ > > > > > > > > > > > Docker Build Test Pipeline (Native): > > > > > > > > > > > https://github.com/apache/kafka/actions/runs/16220242161 > > > > > > > > > > > > > > > > > > > > > > Note: > > > > > > > > > > > As described in the release thread, the native image > > > > > > configuration > > > > > > > > > > > files had to be updated. I'm not able to run the full > > > > > > > > > > > system > > > > > test > > > > > > > > > > > suite with the GraalVM agent to generate the new > > > > configuration > > > > > > files. > > > > > > > > > > > Luke has kindly run a few commands to produce a fix to the > > > > > config > > > > > > > > > > > files. I validated that the kafka-native:4.1.0-rc0 image > > > > passes > > > > > > the > > > > > > > > > > > quickstart (clients, connect and streams), but as that > > > > > > > > > > > image > > > > is > > > > > > only > > > > > > > > > > > for development, I've not tested it any further. > > > > > > > > > > > Also all the other release artifacts were generated before > > > > the > > > > > > fix to > > > > > > > > > > > the native image configuration files was merged. As these > > > > files > > > > > > are > > > > > > > > > > > not part of any of the other release artifacts and as > > > > > > > > > > > this is > > > > > > rc0, I > > > > > > > > > > > deemed that acceptable. > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > Mickael > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > Paolo Patierno > > > > > > > > > > > > > > > > > > > > > > *Senior Principal Software Engineer @ Red Hat**CNCF > > > > Ambassador* > > > > > > > > > > > *Microsoft MVP on **Azure* > > > > > > > > > > > > > > > > > > > > > > Twitter : @ppatierno <http://twitter.com/ppatierno> > > > > > > > > > > > Linkedin : paolopatierno < > > > > > > http://it.linkedin.com/in/paolopatierno> > > > > > > > > > > > GitHub : ppatierno <https://github.com/ppatierno> > > > > > > > > > > > Paolo Patierno > > > > > > > > > > > > > > > > > > > > > > *Senior Principal Software Engineer @ Red Hat**CNCF > > > > Ambassador* > > > > > > > > > > > *Microsoft MVP on **Azure* > > > > > > > > > > > > > > > > > > > > > > Twitter : @ppatierno <http://twitter.com/ppatierno> > > > > > > > > > > > Linkedin : paolopatierno < > > > > > > http://it.linkedin.com/in/paolopatierno> > > > > > > > > > > > GitHub : ppatierno <https://github.com/ppatierno> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Paolo Patierno > > > > > > *Senior Principal Software Engineer @ Red Hat**CNCF Ambassador* > > > *Microsoft MVP on **Azure* > > > > > > Twitter : @ppatierno <http://twitter.com/ppatierno> > > > Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno> > > > GitHub : ppatierno <https://github.com/ppatierno>
