> I am a bit unclear on the precise process regarding what parts of this get merged at what time, and whether the release first needs to be done or not.
The order is as follows: 1. Release approved as part of this vote. After this we follow the steps from here: https://cwiki.apache.org/confluence/display/KAFKA/Release+Process#ReleaseProcess-Afterthevotepasses 2. Upload artifacts to maven etc. These artifacts do not have RC suffix in them. You need a PMC member to mark these artifacts as "production" in apache svn. 3. Update website changes (docs, blog etc.). This is where your PRs on kafka-site repo get merged. 4. Send a release announcement by email. -- Divij Vaidya On Tue, Feb 20, 2024 at 3:02 PM Stanislav Kozlovski <stanis...@confluent.io.invalid> wrote: > Thanks for testing the release! And thanks for the review on the > documentation. Good catch on the license too. > > I have addressed the comments in the blog PR, and opened a few other PRs to > the website in relation to the release. > > - 37: Add download section for the latest 3.7 release > <https://github.com/apache/kafka-site/pull/583/files> > - 37: Update default docs to point to the 3.7.0 release docs > <https://github.com/apache/kafka-site/pull/582> > - 3.7: Add blog post for Kafka 3.7 > <https://github.com/apache/kafka-site/pull/578> > - MINOR: Update stale upgrade_3_6_0 header links in documentation > <https://github.com/apache/kafka-site/pull/580> > - 37: Add upgrade notes for the 3.7.0 release > <https://github.com/apache/kafka-site/pull/581> > > I am a bit unclear on the precise process regarding what parts of this get > merged at what time, and whether the release first needs to be done or not. > > Best, > Stanislav > > On Mon, Feb 19, 2024 at 8:34 PM Divij Vaidya <divijvaidy...@gmail.com> > wrote: > > > Great. In that case we can fix the license issue retrospectively. I have > > created a JIRA for it https://issues.apache.org/jira/browse/KAFKA-16278 > > and > > also updated the release process (which redirects to > > https://issues.apache.org/jira/browse/KAFKA-12622) to check for the > > correct > > license in both the kafka binaries. > > > > I am +1 (binding) assuming Mickael's concerns about update notes to 3.7 > are > > addressed before release. > > > > -- > > Divij Vaidya > > > > > > > > On Mon, Feb 19, 2024 at 6:08 PM Mickael Maison <mickael.mai...@gmail.com > > > > wrote: > > > > > Hi, > > > > > > I agree with Josep, I don't think it's worth making a new RC just for > > this. > > > > > > Thanks Stanislav for sharing the test results. The last thing holding > > > me from casting my vote is the missing upgrade notes for 3.7.0. > > > > > > Thanks, > > > Mickael > > > > > > > > > > > > On Mon, Feb 19, 2024 at 4:28 PM Josep Prat <josep.p...@aiven.io.invalid > > > > > wrote: > > > > > > > > I think I remember finding a similar problem (NOTICE_binary) and it > > > didn't > > > > qualify for an extra RC > > > > > > > > Best, > > > > > > > > On Mon, Feb 19, 2024 at 3:44 PM Divij Vaidya < > divijvaidy...@gmail.com> > > > > wrote: > > > > > > > > > I have performed the following checks. The only thing I would like > to > > > call > > > > > out is the missing licenses before providing a vote. How do we want > > > > > to proceed on this? What have we done in the past? (Creating a new > RC > > > is > > > > > overkill IMO for this license issue). > > > > > > > > > > ## License check > > > > > > > > > > Test: Validate license of dependencies for both 2.12 & 2.13 binary. > > > > > Result: Missing license for some scala* libraries specifically for > > > 2.12. > > > > > Seems like we have been missing these licenses for quite some > version > > > now. > > > > > > > > > > ``` > > > > > for f in $(ls libs | grep -v "^kafka\|connect\|trogdor"); do if ! > > grep > > > -q > > > > > ${f%.*} LICENSE; then echo "${f%.*} is missing in license file"; > fi; > > > done > > > > > scala-collection-compat_2.12-2.10.0 is missing in license file > > > > > scala-java8-compat_2.12-1.0.2 is missing in license file > > > > > scala-library-2.12.18 is missing in license file > > > > > scala-logging_2.12-3.9.4 is missing in license file > > > > > scala-reflect-2.12.18 is missing in license file > > > > > ``` > > > > > > > > > > ## Long running tests for memory leak (on ARM machine with zstd) > > > > > > > > > > Test: Run produce/consume for a few hours and verify no gradual > > > increase in > > > > > heap. > > > > > Result: No heap increase observed. The overall CPU utilization is > > lower > > > > > compared to 3.5.1. > > > > > > > > > > ## Verify system test results > > > > > > > > > > Test: Spot check the results of system tests. > > > > > Result: I have verified that the system tests are passing across > > > different > > > > > runs. > > > > > > > > > > -- > > > > > Divij Vaidya > > > > > > > > > > > > > > > > > > > > On Sun, Feb 18, 2024 at 2:50 PM Stanislav Kozlovski > > > > > <stanis...@confluent.io.invalid> wrote: > > > > > > > > > > > The latest system test build completed successfully - > > > > > > > > > > > > > > > > > > > > > > > https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1708250728--apache--3.7--02197edaaa/2024-02-18--001./2024-02-18--001./report.html > > > > > > > > > > > > *System tests are therefore all good*. We just have some flakes > > > > > > > > > > > > On Sun, Feb 18, 2024 at 10:45 AM Stanislav Kozlovski < > > > > > > stanis...@confluent.io> > > > > > > wrote: > > > > > > > > > > > > > The upgrade test passed -> > > > > > > > > > > > > > > > > > > > > > > > > https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1708103771--apache--3.7--bb6990114b/2024-02-16--001./2024-02-16--001./report.html > > > > > > > > > > > > > > The replica verification test succeeded in ZK mode, but failed > in > > > > > > > ISOLATED_KRAFT. It just seems to be very flaky. > > > > > > > > > > > > > > > > > > > > > > > > https://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/system-test-kafka-branch-builder--1708100119--apache--3.7--bb6990114b/2024-02-16--001./2024-02-16--001./report.html > > > > > > > > > > > > > > Scheduling another run in > > > > > > > > > > > > > > > > https://jenkins.confluent.io/job/system-test-kafka-branch-builder/6062/ > > > > > > > > > > > > > > On Fri, Feb 16, 2024 at 6:39 PM Stanislav Kozlovski < > > > > > > > stanis...@confluent.io> wrote: > > > > > > > > > > > > > >> Thanks all for the help in verifying. > > > > > > >> > > > > > > >> I have updated > > > > > > >> > > > > > > > > > > > > > > > > > https://gist.github.com/stanislavkozlovski/820976fc7bfb5f4dcdf9742fd96a9982 > > > > > > >> with the system tests. > > > > > > >> There were two builds ran, and across those - the following > > tests > > > > > failed > > > > > > >> two times in a row: > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > *kafkatest.tests.tools.replica_verification_test.ReplicaVerificationToolTest#test_replica_lagsArguments:{ > > > > > > >> "metadata_quorum": "ZK"}*Fails with the same error of > > > > > > >> *`TimeoutError('Timed out waiting to reach non-zero number of > > > replica > > > > > > >> lags.')`* > > > > > > >> I have scheduled a re-run of this specific test here -> > > > > > > >> > > > > > > > https://jenkins.confluent.io/job/system-test-kafka-branch-builder/6057 > > > > > > >> > > > > > > >> > > > *kafkatest.tests.core.upgrade_test.TestUpgrade#test_upgradeArguments:{ > > > > > > >> "compression_types": [ "zstd" ], "from_kafka_version": > "2.4.1", > > > > > > >> "to_message_format_version": null}* > > > > > > >> Fails with the same error of > > > > > > >> *`TimeoutError('Producer failed to produce messages for > 20s.')`* > > > > > > >> > > > *kafkatest.tests.core.upgrade_test.TestUpgrade#test_upgradeArguments:{ > > > > > > >> "compression_types": [ "lz4" ], "from_kafka_version": "3.0.2", > > > > > > >> "to_message_format_version": null}* > > > > > > >> Fails with the same error of *`TimeoutError('Producer failed > to > > > > > produce > > > > > > >> messages for 20s.')`* > > > > > > >> > > > > > > >> I have scheduled a re-run of this test here -> > > > > > > >> > > > > > > > > > https://jenkins.confluent.io/job/system-test-kafka-branch-builder/6058/ > > > > > > >> > > > > > > >> On Fri, Feb 16, 2024 at 12:15 PM Vedarth Sharma < > > > > > > vedarth.sha...@gmail.com> > > > > > > >> wrote: > > > > > > >> > > > > > > >>> Hey Stanislav, > > > > > > >>> > > > > > > >>> Thanks for the release candidate. > > > > > > >>> > > > > > > >>> +1 (non-binding) > > > > > > >>> > > > > > > >>> I tested and verified the docker image artifact > > > > > > apache/kafka:3.7.0-rc4:- > > > > > > >>> - verified create topic, produce messages and consume > messages > > > flow > > > > > > when > > > > > > >>> running the docker image with > > > > > > >>> - default configs > > > > > > >>> - configs provided via env variables > > > > > > >>> - configs provided via file input > > > > > > >>> - verified the html documentation for docker image. > > > > > > >>> - ran the example docker compose files successfully. > > > > > > >>> > > > > > > >>> All looks good for the docker image artifact! > > > > > > >>> > > > > > > >>> Thanks and regards, > > > > > > >>> Vedarth > > > > > > >>> > > > > > > >>> > > > > > > >>> On Thu, Feb 15, 2024 at 10:58 PM Mickael Maison < > > > > > > >>> mickael.mai...@gmail.com> > > > > > > >>> wrote: > > > > > > >>> > > > > > > >>> > Hi Stanislav, > > > > > > >>> > > > > > > > >>> > Thanks for running the release. > > > > > > >>> > > > > > > > >>> > I did the following testing: > > > > > > >>> > - verified the check sums and signatures > > > > > > >>> > - ran ZooKeeper and KRaft quickstarts with Scala 2.13 > > binaries > > > > > > >>> > - ran a successful migration from ZooKeeper to KRaft > > > > > > >>> > > > > > > > >>> > We seem to be missing the upgrade notes for 3.7.0 in the > > docs. > > > See > > > > > > >>> > https://kafka.apache.org/37/documentation.html#upgrade > that > > > still > > > > > > >>> > points to 3.6.0 > > > > > > >>> > Before voting I'd like to see results from the system tests > > > too. > > > > > > >>> > > > > > > > >>> > Thanks, > > > > > > >>> > Mickael > > > > > > >>> > > > > > > > >>> > On Thu, Feb 15, 2024 at 6:06 PM Andrew Schofield > > > > > > >>> > <andrew_schofield_j...@outlook.com> wrote: > > > > > > >>> > > > > > > > > >>> > > +1 (non-binding). I used the staged binaries with Scala > > > 2.13. I > > > > > > tried > > > > > > >>> > the new group coordinator > > > > > > >>> > > and consumer group protocol which is included with the > > Early > > > > > Access > > > > > > >>> > release of KIP-848. > > > > > > >>> > > Also verified the availability of the new APIs. All > working > > > as > > > > > > >>> expected. > > > > > > >>> > > > > > > > > >>> > > Thanks, > > > > > > >>> > > Andrew > > > > > > >>> > > > > > > > > >>> > > > On 15 Feb 2024, at 05:07, Paolo Patierno < > > > > > > paolo.patie...@gmail.com > > > > > > >>> > > > > > > > >>> > wrote: > > > > > > >>> > > > > > > > > > >>> > > > +1 (non-binding). I used the staged binaries with Scala > > > 2.13 > > > > > and > > > > > > >>> mostly > > > > > > >>> > > > focused on the ZooKeeper to KRaft migration with > multiple > > > > > tests. > > > > > > >>> > Everything > > > > > > >>> > > > works fine. > > > > > > >>> > > > > > > > > > >>> > > > Thanks > > > > > > >>> > > > Paolo > > > > > > >>> > > > > > > > > > >>> > > > On Mon, 12 Feb 2024, 22:06 Jakub Scholz, < > > ja...@scholz.cz> > > > > > > wrote: > > > > > > >>> > > > > > > > > > >>> > > >> +1 (non-binding). I used the staged binaries with > Scala > > > 2.13 > > > > > and > > > > > > >>> the > > > > > > >>> > staged > > > > > > >>> > > >> Maven artifacts to run my tests. All seems to work > fine. > > > > > Thanks. > > > > > > >>> > > >> > > > > > > >>> > > >> Jakub > > > > > > >>> > > >> > > > > > > >>> > > >> On Fri, Feb 9, 2024 at 4:20 PM Stanislav Kozlovski > > > > > > >>> > > >> <stanis...@confluent.io.invalid> wrote: > > > > > > >>> > > >> > > > > > > >>> > > >>> Hello Kafka users, developers and client-developers, > > > > > > >>> > > >>> > > > > > > >>> > > >>> This is the second candidate we are considering for > > > release > > > > > of > > > > > > >>> Apache > > > > > > >>> > > >> Kafka > > > > > > >>> > > >>> 3.7.0. > > > > > > >>> > > >>> > > > > > > >>> > > >>> Major changes include: > > > > > > >>> > > >>> - Early Access to KIP-848 - the next generation of > the > > > > > consumer > > > > > > >>> > rebalance > > > > > > >>> > > >>> protocol > > > > > > >>> > > >>> - Early Access to KIP-858: Adding JBOD support to > KRaft > > > > > > >>> > > >>> - KIP-714: Observability into Client metrics via a > > > > > standardized > > > > > > >>> > interface > > > > > > >>> > > >>> > > > > > > >>> > > >>> Release notes for the 3.7.0 release: > > > > > > >>> > > >>> > > > > > > >>> > > >>> > > > > > > >>> > > >> > > > > > > >>> > > > > > > > >>> > > > > > > > > > > > > > > > > > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/RELEASE_NOTES.html > > > > > > >>> > > >>> > > > > > > >>> > > >>> *** Please download, test and vote by Thursday, > > February > > > > > 15th, > > > > > > >>> 9AM > > > > > > >>> > PST > > > > > > >>> > > >> *** > > > > > > >>> > > >>> > > > > > > >>> > > >>> Kafka's KEYS file containing PGP keys we use to sign > > the > > > > > > release: > > > > > > >>> > > >>> https://kafka.apache.org/KEYS > > > > > > >>> > > >>> > > > > > > >>> > > >>> * Release artifacts to be voted upon (source and > > binary): > > > > > > >>> > > >>> > > > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/ > > > > > > >>> > > >>> > > > > > > >>> > > >>> * Docker release artifact to be voted upon: > > > > > > >>> > > >>> apache/kafka:3.7.0-rc4 > > > > > > >>> > > >>> > > > > > > >>> > > >>> * Maven artifacts to be voted upon: > > > > > > >>> > > >>> > > > > > > >>> > > > > > > > > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > >>> > > >>> > > > > > > >>> > > >>> * Javadoc: > > > > > > >>> > > >>> > > > > > > >>> > > > https://home.apache.org/~stanislavkozlovski/kafka-3.7.0-rc4/javadoc/ > > > > > > >>> > > >>> > > > > > > >>> > > >>> * Tag to be voted upon (off 3.7 branch) is the 3.7.0 > > tag: > > > > > > >>> > > >>> > https://github.com/apache/kafka/releases/tag/3.7.0-rc4 > > > > > > >>> > > >>> > > > > > > >>> > > >>> * Documentation: > > > > > > >>> > > >>> https://kafka.apache.org/37/documentation.html > > > > > > >>> > > >>> > > > > > > >>> > > >>> * Protocol: > > > > > > >>> > > >>> https://kafka.apache.org/37/protocol.html > > > > > > >>> > > >>> > > > > > > >>> > > >>> * Successful Jenkins builds for the 3.7 branch: > > > > > > >>> > > >>> > > > > > > >>> > > >>> Unit/integration tests: I am in the process of > running > > > and > > > > > > >>> analyzing > > > > > > >>> > > >> these. > > > > > > >>> > > >>> System tests: I am in the process of running these. > > > > > > >>> > > >>> > > > > > > >>> > > >>> Expect a follow-up over the weekend > > > > > > >>> > > >>> > > > > > > >>> > > >>> * Successful Docker Image Github Actions Pipeline for > > 3.7 > > > > > > branch: > > > > > > >>> > > >>> Docker Build Test Pipeline: > > > > > > >>> > > >>> > > https://github.com/apache/kafka/actions/runs/7845614846 > > > > > > >>> > > >>> > > > > > > >>> > > >>> /************************************** > > > > > > >>> > > >>> > > > > > > >>> > > >>> Best, > > > > > > >>> > > >>> Stanislav > > > > > > >>> > > >>> > > > > > > >>> > > >> > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > > >> > > > > > > >> > > > > > > >> -- > > > > > > >> Best, > > > > > > >> Stanislav > > > > > > >> > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best, > > > > > > > Stanislav > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Best, > > > > > > Stanislav > > > > > > > > > > > > > > > > > > > > > > > -- > > > > [image: Aiven] <https://www.aiven.io> > > > > > > > > *Josep Prat* > > > > Open Source Engineering Director, *Aiven* > > > > josep.p...@aiven.io | +491715557497 > > > > aiven.io <https://www.aiven.io> | < > > > https://www.facebook.com/aivencloud> > > > > <https://www.linkedin.com/company/aiven/> < > > > https://twitter.com/aiven_io> > > > > *Aiven Deutschland GmbH* > > > > Alexanderufer 3-7, 10117 Berlin > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > -- > Best, > Stanislav >
