Hi David, I've tested the RC3 and this is what I've done so far and what I've found:
- build from src kafka-4.0.0-src.tgz with Java 17 - run unit test and integration tests - run client examples from the examples module (consume/produce/tx) - build clients with Java 11. Run client unit tests, integration tests, and examples as above - Followed the quickstart guide to start a broker - Played with command line tools to consume, produce, manage topics - Tried out the new regex validator tool All good with the above steps, with these findings: - When running unit and integration tests on the downloaded kafka-4.0.0-src.tgz, I got some failures that recovered as TengYao mentioned, but I noticed a persistent failure locally, in MetadataSchemaCheckerToolTest.testVerifyEvolutionGit. This test was added in 4.0, and requires a git repo setup, so fails with: *java.lang.RuntimeException: Invalid directory, need to be within a Git repository*. This means that building and running the tests straight from a downloaded kafka src folder doesn't work anymore. It used to work with previous versions (ie. 3.9, I checked again just in case), because this test didn't exist. If this was intentional I would expect we need to explain it somewhere, but can't find anything about it. Am I missing something? - The quickstart https://kafka.apache.org/quickstart still has sections to start/stop Zookeeper (same for Streams Quickstart). - Also the quickstart has invalid commands to format log directories and start the server (pointing to wrong server.properties location under /kraft). It's OK on the README. Best, Lianet On Wed, Mar 12, 2025 at 10:41 AM David Jacot <david.ja...@gmail.com> wrote: > Hi all, > > All the documentation blockers are now closed. Let's continue the > validation process. I don't expect to cut a new RC unless we find a > non-documentation blocker. > > Best, > David > > On Wed, Mar 12, 2025 at 3:37 PM David Jacot <david.ja...@gmail.com> wrote: > > > > Awesome, thanks! I just merged it. > > > > Best, > > David > > > > On Wed, Mar 12, 2025 at 3:33 PM 黃竣陽 <s7133...@gmail.com> wrote: > > > > > > Hello David, > > > > > > FYI, https://github.com/apache/kafka-site/pull/669 this PR resolve > “1.7 Compatibility” section is empty. > > > > > > Best, > > > Jiunn-Yang > > > > > > > David Jacot <david.ja...@gmail.com> 於 2025年3月12日 晚上10:15 寫道： > > > > > > > > Hi all, > > > > > > > > FYI - I just updated the documentation. The "1.7 Compatibility" > > > > section is empty. Looking into it. > > > > > > > > Best, > > > > David > > > > > > > > On Wed, Mar 12, 2025 at 1:51 PM David Jacot > <dja...@confluent.io.invalid> wrote: > > > >> > > > >> Hi, > > > >> > > > >> Thanks for pointing that out. I will update the documentation by > the end of > > > >> the day. > > > >> > > > >> Best, > > > >> David > > > >> > > > >> On Wed, Mar 12, 2025 at 1:15 PM TengYao Chi <kiting...@gmail.com> > wrote: > > > >> > > > >>> Hi Federico > > > >>> > > > >>> The issue you mentioned has been resolved by > > > >>> https://github.com/apache/kafka/pull/19146 > > > >>> > > > >>> Best, > > > >>> TengYao > > > >>> > > > >>> > > > >>> Federico Valeri <fedeval...@gmail.com> 於 2025年3月12日 週三 下午7:32寫道： > > > >>> > > > >>>> Hi David, in the doc menu I see the item "1.6 Differences Between > > > >>>> KRaft mode and ZooKeeper mode", but paragraph 1.6 talks about > Docker. > > > >>>> Is this supposed to be there? > > > >>>> > > > >>>> On Wed, Mar 12, 2025 at 10:56 AM Luke Chen <show...@gmail.com> > wrote: > > > >>>>> > > > >>>>> Hi David, > > > >>>>> > > > >>>>> Thanks for the analysis. > > > >>>>> > > > >>>>>> * kafkatest.tests.connect.connect_distributed_test - These > tests are > > > >>>> flaky > > > >>>>> in our test infrastructure but they pass locally. > > > >>>>> > > > >>>>> If they can be passed locally, it should be fine. > > > >>>>> > > > >>>>>> * kafkatest.tests.core.network_degrade_test - This is not a > blocker > > > >>> as > > > >>>> it > > > >>>>> does not test any Kafka bits besides Trogdor. > > > >>>>> > > > >>>>> Hmm... from the test description, it is indeed not related to > kafka > > > >>>> itself. > > > >>>>> But since we did huge change in this release, I'd like to be more > > > >>>> careful. > > > >>>>> I'll try to run this test suite in our internal environment > again and > > > >>>>> report the results here. > > > >>>>> > > > >>>>>> * kafkatest.tests.tools.replica_verification_test - It has been > > > >>> failing > > > >>>>> for > > > >>>>> a while. The tool is deprecated and not really used so I don't > consider > > > >>>> it > > > >>>>> as a blocker. However, there is already a PR inflight to improve > it. > > > >>>>> > > > >>>>> Sounds good to me. > > > >>>>> > > > >>>>> > > > >>>>> Thanks. > > > >>>>> Luke > > > >>>>> > > > >>>>> On Tue, Mar 11, 2025 at 6:16 PM David Jacot > > > >>> <dja...@confluent.io.invalid > > > >>>>> > > > >>>>> wrote: > > > >>>>> > > > >>>>>> Hi Luke, > > > >>>>>> > > > >>>>>> +1 for backporting https://github.com/apache/kafka/pull/19172. > > > >>>>>> > > > >>>>>> Regarding the system tests, here is my take: > > > >>>>>> * kafkatest.tests.connect.connect_distributed_test - These > tests are > > > >>>> flaky > > > >>>>>> in our test infrastructure but they pass locally. > > > >>>>>> * kafkatest.tests.core.network_degrade_test - This is not a > blocker > > > >>> as > > > >>>> it > > > >>>>>> does not test any Kafka bits besides Trogdor. > > > >>>>>> * kafkatest.tests.tools.replica_verification_test - It has been > > > >>>> failing for > > > >>>>>> a while. The tool is deprecated and not really used so I don't > > > >>>> consider it > > > >>>>>> as a blocker. However, there is already a PR inflight to > improve it. > > > >>>>>> > > > >>>>>> Overall, system tests look pretty good to me. Do you agree? > > > >>>>>> > > > >>>>>> Best, > > > >>>>>> David > > > >>>>>> > > > >>>>>> On Tue, Mar 11, 2025 at 8:53 AM Luke Chen <show...@gmail.com> > wrote: > > > >>>>>> > > > >>>>>>> Hi David, > > > >>>>>>> > > > >>>>>>> I saw there are some failed tests in system test > > > >>>>>>> < > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > https://confluent-open-source-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/trunk/2025-03-06--001.ccab7b98-edaa-40c5-b33d-de1650ea767c--1741313025--apache--4.0--43897f84ac/report.html > > > >>>>>>>> , > > > >>>>>>> what does that mean for this RC? > > > >>>>>>> Are they blockers? Should we file JIRA issues to fix them? > > > >>>>>>> > > > >>>>>>> Also, I'd like to backport this upgrade doc improvement PR > > > >>>>>>> <https://github.com/apache/kafka/pull/19172>to 4.0 branch if > you > > > >>>> don't > > > >>>>>>> have > > > >>>>>>> objections. > > > >>>>>>> > > > >>>>>>> Thanks. > > > >>>>>>> Luke > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> On Mon, Mar 10, 2025 at 5:52 PM David Jacot > > > >>>> <dja...@confluent.io.invalid > > > >>>>>>> > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>>> Hi Chia-Ping, > > > >>>>>>>> > > > >>>>>>>> Thanks. I will update the documentation soon. > > > >>>>>>>> > > > >>>>>>>> Best, > > > >>>>>>>> David > > > >>>>>>>> > > > >>>>>>>> On Sun, Mar 9, 2025 at 2:43 AM Chia-Ping Tsai < > > > >>> chia7...@apache.org > > > >>>>> > > > >>>>>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> hi David > > > >>>>>>>>> > > > >>>>>>>>> I have cherry-pick > https://github.com/apache/kafka/pull/19146 > > > >>>> to 4.0 > > > >>>>>>> to > > > >>>>>>>>> fix the link of https://kafka.apache.org/40/zk2kraft > > > >>>>>>>>> > > > >>>>>>>>> Best, > > > >>>>>>>>> Chia-Ping > > > >>>>>>>>> > > > >>>>>>>>> On 2025/03/08 10:08:24 David Jacot wrote: > > > >>>>>>>>>> Hello Kafka users, developers and client-developers, > > > >>>>>>>>>> > > > >>>>>>>>>> This is the third candidate for release of Apache Kafka > > > >>> 4.0.0. > > > >>>>>>>>>> > > > >>>>>>>>>> - This is the first release without Apache Zookeeper > > > >>>>>>>>>> - The Next Generation of the Consumer Rebalance Protocol is > > > >>>>>> Generally > > > >>>>>>>>> Available > > > >>>>>>>>>> - The Transactions Server-Side Defense (Phase 2) is > Generally > > > >>>>>>> Available > > > >>>>>>>>>> - Queues for Kafka is in Early Access > > > >>>>>>>>>> - Kafka uses log4j2 > > > >>>>>>>>>> - Drop broker and tools support for Java 11 > > > >>>>>>>>>> - Remove old client protocol API versions > > > >>>>>>>>>> > > > >>>>>>>>>> Release notes for the 4.0.0 release: > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc3/RELEASE_NOTES.html > > > >>>>>>>>>> > > > >>>>>>>>>> *** Please download, test and vote by Wed, March 12 9am PT > > > >>>>>>>>>> > > > >>>>>>>>>> Kafka's KEYS file containing PGP keys we use to sign the > > > >>>> release: > > > >>>>>>>>>> https://kafka.apache.org/KEYS > > > >>>>>>>>>> > > > >>>>>>>>>> * Release artifacts to be voted upon (source and binary): > > > >>>>>>>>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc3/ > > > >>>>>>>>>> > > > >>>>>>>>>> * Docker release artifacts to be voted upon: > > > >>>>>>>>>> apache/kafka:4.0.0-rc3 > > > >>>>>>>>>> apache/kafka-native:4.0.0-rc3 > > > >>>>>>>>>> > > > >>>>>>>>>> * Maven artifacts to be voted upon: > > > >>>>>>>>>> > > > >>>>>>> > > > >>>> > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > >>>>>>>>>> > > > >>>>>>>>>> * Javadoc: > > > >>>>>>>>>> > > > >>>> https://dist.apache.org/repos/dist/dev/kafka/4.0.0-rc3/javadoc/ > > > >>>>>>>>>> > > > >>>>>>>>>> * Tag to be voted upon (off 4.0 branch) is the 4.0.0 tag: > > > >>>>>>>>>> https://github.com/apache/kafka/releases/tag/4.0.0-rc3 > > > >>>>>>>>>> > > > >>>>>>>>>> * Documentation: > > > >>>>>>>>>> https://kafka.apache.org/40/documentation.html > > > >>>>>>>>>> > > > >>>>>>>>>> * Protocol: > > > >>>>>>>>>> https://kafka.apache.org/40/protocol.html > > > >>>>>>>>>> > > > >>>>>>>>>> * Successful CI builds for the 4.0 branch: > > > >>>>>>>>>> Unit/integration tests: > > > >>>>>>>>> https://github.com/apache/kafka/actions/runs/13730048850 > > > >>>>>>>>>> System tests: > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > https://confluent-open-source-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/trunk/2025-03-06--001.ccab7b98-edaa-40c5-b33d-de1650ea767c--1741313025--apache--4.0--43897f84ac/report.html > > > >>>>>>>>>> > > > >>>>>>>>>> * Successful Docker Image Github Actions Pipeline for 4.0 > > > >>>> branch: > > > >>>>>>>>>> Docker Build Test Pipeline (JVM): > > > >>>>>>>>>> https://github.com/apache/kafka/actions/runs/13735818889 > > > >>>>>>>>>> Docker Build Test Pipeline (Native): > > > >>>>>>>>>> https://github.com/apache/kafka/actions/runs/13735821944 > > > >>>>>>>>>> > > > >>>>>>>>>> /************************************** > > > >>>>>>>>>> > > > >>>>>>>>>> Thanks, > > > >>>>>>>>>> David Jacot > > > >>>>>>>>>> > > > >>>>>>>>>> PS: I used RC3 instead of RC2 by mistake. > > > >>>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>> > > > >>> > > > >
