Hi Luke, I took a look at getting 3.9 to run Java 23 on CI in this PR https://github.com/apache/kafka/pull/19311 which is just 3.9 plus all the other backport PRs, plus a few extra backports and changes to get Java 23 to build locally. I was planning to raise PRs for these bits individually once I had a build working.
But I am roadblocked on getting Jenkins to actually use Java 23. The CI run appears to not have picked up the change (see https://github.com/apache/kafka/pull/19311/files#diff-e6ffa5dc854b843b3ee3c3c28f8eae2f436c2df2b1ca299cca1fa5982e390cf8) to the script I did in the last commit. I'm not familiar enough with 3.9's CI setup to know how to fix it, and I also don't think I have the necessary access to Jenkins to go dig into the job configuration to figure it out. In particular I don't know if tools { jdk 'jdk_23_latest' } is right, because I don't know which JDKs are available on Jenkins, and without admin access, I don't believe I can get a list anywhere. If someone has full access to Jenkins, maybe they can take a look? Den ons. 2. apr. 2025 kl. 14.04 skrev Dejan Stojadinović < dejan2...@gmail.com>: > Hi all, > > Just keep in mind that *Gradle version 8.14 *(currently under development) > will support *Java 24*: > > - https://docs.gradle.org/8.13/userguide/compatibility.html > - https://github.com/gradle/gradle/issues/32290#issuecomment-2741096604 > - https://github.com/gradle/gradle/issues/32289#issuecomment-2741056207 > > > > On Wed, Apr 2, 2025 at 1:25 PM Luke Chen <show...@gmail.com> wrote: > > > Hi Stig and all, > > > > For backporting the change to 3.9, if there's no objections, I think we > can > > start to merge them tomorrow. > > For the Java 24 testing, it's fine we ship 3.9 without Java 24 test if > > gradle release is delayed since this is also the state of what Kafka v4.0 > > is. > > > > So we should firstly test Java 23 in CI for 3.9. @Stig, could you help > open > > a PR for it? > > > > Thanks. > > Luke > > > > On Sat, Mar 29, 2025 at 2:29 AM Stig Rohde Døssing < > stigdoess...@gmail.com > > > > > wrote: > > > > > Thanks. I'll wait for that PR then. Since it's blocked by Gradle, and > the > > > plan is for Gradle to release a new version on April 7th (see > > > https://github.com/gradle/gradle/issues/32289#issuecomment-2741056207 > ), > > > maybe the way to go is to aim to test with Java 23 on 3.9 for now? Once > > we > > > can test with Java 24 on trunk, maybe we can consider backporting the > > > necessary changes to 3.9 to get that branch tested against Java 24 as > > well, > > > and getting it running with Java 23 at least gets us closer. > > > > > > Regarding CI, I made an attempt at getting Jenkins to run with Java 23 > > here > > > > > > > > > https://github.com/apache/kafka/pull/19311/commits/f4cd8d5d664a62b280c0d2ea1bc731069443ba87 > > > , > > > but it doesn't look like CI is picking up the change. Maybe Jenkins is > > > configured to fetch the script from a specific branch? I can't tell > > since I > > > don't have access to the job configurations. > > > > > > Den fre. 28. mar. 2025 kl. 19.00 skrev Chia-Ping Tsai < > > chia7...@gmail.com > > > >: > > > > > > > hi Stig > > > > > > > > > Does anyone know where the corresponding CI script > > > > files are located in 3.9? > > > > > > > > https://github.com/apache/kafka/blob/3.9/Jenkinsfile#L165 > > > > > > > > > I noticed that trunk isn't currently testing against Java 24. I'll > > try > > > to > > > > make a PR to fix that, which will probably require another bump of > > > > Spotbugs, which may prompt another backport PR. > > > > > > > > there is already a PR (https://github.com/apache/kafka/pull/19235) > > > > > > > > Best, > > > > Chia-Ping > > > > > > > > Stig Rohde Døssing <stigdoess...@gmail.com> 於 2025年3月29日 週六 > 上午12:02寫道： > > > > > > > > > Chia-Ping, I think we should aim for Java 24 support, since 23 > > reaches > > > > end > > > > > of life in a month or so. > > > > > > > > > > Here are PRs for 4 of the changes linked at > > > > > https://issues.apache.org/jira/browse/KAFKA-17638 > > > > > > > > > > Scala upgrade https://github.com/apache/kafka/pull/19307 > > > > > Compiler warning 1 https://github.com/apache/kafka/pull/19308 > > > > > Compiler warning 2 https://github.com/apache/kafka/pull/19309 > > > > > Spotbugs update https://github.com/apache/kafka/pull/19310 > > > > > > > > > > The missing backports after that will be > > > > > https://github.com/apache/kafka/pull/17409 and > > > > > https://github.com/apache/kafka/pull/17403 > > > > > > > > > > For the former PR, it cherry picks mostly cleanly, but it won't > pass > > > > tests > > > > > until the other changes are merged. The CI script changes are done > to > > > > files > > > > > that don't exist in 3.9. Does anyone know where the corresponding > CI > > > > script > > > > > files are located in 3.9? > > > > > > > > > > The latter PR contains only documentation changes calling out that > > Java > > > > 23 > > > > > is now supported. I think we are better off adjusting the docs on > > trunk > > > > to > > > > > say that Java 24 is supported as of 3.9.1 instead of backporting > that > > > > > change. > > > > > > > > > > I noticed that trunk isn't currently testing against Java 24. I'll > > try > > > to > > > > > make a PR to fix that, which will probably require another bump of > > > > > Spotbugs, which may prompt another backport PR. > > > > > > > > > > Den fre. 28. mar. 2025 kl. 15.08 skrev Chia-Ping Tsai < > > > > chia7...@gmail.com > > > > > >: > > > > > > > > > > > hi Stig > > > > > > > > > > > > We should run CI under Java 23 if we want to make 3.9 support > Java > > 23 > > > > > > officially, so +1 to backport them > > > > > > > > > > > > Best, > > > > > > Chia-Ping > > > > > > > > > > > > > > > > > > > Mickael Maison <mickael.mai...@gmail.com> 於 2025年3月28日 下午6:43 > > 寫道： > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > I'm also supportive of backporting this change to the 3.9 > branch. > > > > > > > > > > > > > > Thanks, > > > > > > > Mickael > > > > > > > > > > > > > > > > > > > > > > > > > > > >> On Fri, Mar 28, 2025 at 10:16 AM Stig Rohde Døssing > > > > > > >> <stigdoess...@gmail.com> wrote: > > > > > > >> > > > > > > >> Pulling in a discussion from the PR thread: > > > > > > >> > > > > > > >>> There were other PRs that were part of Java 23 support > > generally, > > > > > > should > > > > > > >> these also be backported? > > > > > > https://issues.apache.org/jira/browse/KAFKA-17638 > > > > > > >> > > > > > > >> Greg, thanks for letting me know about these, I was not aware. > > > > > > >> > > > > > > >> I think most likely these changes aren't necessary to allow > > Kafka > > > to > > > > > > run on > > > > > > >> Java 23. I believe upgrading Scala is necessary because the > > > bundled > > > > > ASM > > > > > > >> doesn't understand Java 23 bytecode, and the remaining changes > > > seem > > > > to > > > > > > >> relate to either fixing compiler warnings or adding Java 23 to > > the > > > > > build > > > > > > >> matrix. I think backporting these would be a fine idea, but I > > > don't > > > > > > think > > > > > > >> they're blockers for using Kafka on Java 23 or 24. > > > > > > >> > > > > > > >> I'm happy to open PRs to backport these as well (though for > the > > > > build > > > > > > >> matrix, we should use Java 24 now), assuming there's no strong > > > > > > opposition > > > > > > >> to that? > > > > > > >> > > > > > > >> Den fre. 28. mar. 2025 kl. 05.02 skrev Greg Harris > > > > > > >> <greg.har...@aiven.io.invalid>: > > > > > > >> > > > > > > >>> Hi All, > > > > > > >>> > > > > > > >>> I am still strongly in support of backporting this patch as I > > > > stated > > > > > > in the > > > > > > >>> earlier discussion thread. > > > > > > >>> > > > > > > >>> Thank you Stig, Anton, Danish, Mateusz, Manfred, Monica, > > Istvan, > > > > > > Vincent, > > > > > > >>> Clement, Anshu, Foivos, Drakgoku, Severin, Ozan, Georgios, > and > > > > > > Guillaume, > > > > > > >>> members of our extended community who have commented to > discuss > > > the > > > > > > >>> impact and their support for this patch. > > > > > > >>> I inevitably missed some people, such as everyone who has > been > > > > > > encountering > > > > > > >>> and silently working around the problem just from reading > these > > > > > > threads and > > > > > > >>> GitHub issues. > > > > > > >>> > > > > > > >>> Through our inaction, we have obligated several users to take > > > > actions > > > > > > to > > > > > > >>> work around us: > > > > > > >>> * The Trino project dropped two of their products [1, 2] > > > > > > >>> * The Quarkus project disabled some tests [3] > > > > > > >>> * Quarkus project users downgraded their java version [4] > > > > > > >>> * Spring Boot users have installed the system property > > workaround > > > > [5] > > > > > > >>> > > > > > > >>> In my opinion this is unacceptable, and it's time for us to > fix > > > > this. > > > > > > >>> > > > > > > >>> Thank you Stig for restarting the conversation! > > > > > > >>> Greg > > > > > > >>> > > > > > > >>> [1] https://github.com/trinodb/trino/issues/24419 > > > > > > >>> [2] https://github.com/trinodb/trino/issues/24417 > > > > > > >>> [3] https://github.com/quarkusio/quarkus/pull/43543 > > > > > > >>> [4] > > > > > > >>> > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > https://github.com/anshupitlia/product-information-system/commit/2c3a8dbd974dce0273f74969ec64b661abafef62 > > > > > > >>> [5] > > > > > > >>> > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > https://github.com/vividus-framework/vividus-build-system/commit/53b7016a0d3b0ba04a23b4b1892e1cf7f62ba0a5 > > > > > > >>> > > > > > > >>>> On Thu, Mar 27, 2025 at 7:39 PM Luke Chen < > show...@gmail.com> > > > > > wrote: > > > > > > >>> > > > > > > >>>> Hi Stig, > > > > > > >>>> > > > > > > >>>> Thanks for bringing this to us. > > > > > > >>>> I'm +1 for backporting to 3.9 branch since there's no > > workaround > > > > for > > > > > > Java > > > > > > >>>> 24. > > > > > > >>>> > > > > > > >>>> Thanks. > > > > > > >>>> Luke > > > > > > >>>> > > > > > > >>>> > > > > > > >>>> > > > > > > >>>> On Tue, Mar 18, 2025 at 1:14 AM Stig Rohde Døssing < > > > > > > >>> stigdoess...@gmail.com > > > > > > >>>>> > > > > > > >>>> wrote: > > > > > > >>>> > > > > > > >>>>> Thanks Ismail, > > > > > > >>>>> > > > > > > >>>>> I've opened https://github.com/apache/kafka/pull/19221 > just > > to > > > > get > > > > > > any > > > > > > >>>>> test > > > > > > >>>>> failures out of the way in case it is decided to do this > > > > backport. > > > > > > >>>>> > > > > > > >>>>> I'm hoping people will weigh in with their concerns in this > > > > thread > > > > > if > > > > > > >>>> they > > > > > > >>>>> don't like the idea of backporting this change. > > > > > > >>>>> > > > > > > >>>>> Den man. 17. mar. 2025 kl. 16.43 skrev Ismael Juma < > > > > > > m...@ismaeljuma.com > > > > > > >>>> : > > > > > > >>>>> > > > > > > >>>>>> Hi Stig, > > > > > > >>>>>> > > > > > > >>>>>> Kafka 4.0 is likely to be released in a day or two. Even > > so, I > > > > > think > > > > > > >>> it > > > > > > >>>>>> makes sense to revive the backporting thread given the > lack > > of > > > > > > >>>> workaround > > > > > > >>>>>> for Java 24. > > > > > > >>>>>> > > > > > > >>>>>> Ismael > > > > > > >>>>>> > > > > > > >>>>>> On Mon, Mar 17, 2025 at 7:44 AM Stig Rohde Døssing < > > > > > > >>>>> stigdoess...@gmail.com > > > > > > >>>>>>> > > > > > > >>>>>> wrote: > > > > > > >>>>>> > > > > > > >>>>>>> Hi, > > > > > > >>>>>>> > > > > > > >>>>>>> Some months ago, a reflective shim was added in > > > > > > >>>>>>> https://issues.apache.org/jira/browse/KAFKA-17078, in > > order > > > to > > > > > > >>>> support > > > > > > >>>>>>> running Kafka with SASL on JDKs that no longer support > the > > > > > security > > > > > > >>>>>>> manager. > > > > > > >>>>>>> > > > > > > >>>>>>> This shim was added only to Kafka 4.0, but backporting > was > > > > > > >>> discussed > > > > > > >>>> in > > > > > > >>>>>>> > > > > https://lists.apache.org/thread/vl43q9wqq4xs67xx61f0t0850y2b037o > > > > > . > > > > > > >>>>> There > > > > > > >>>>>>> was > > > > > > >>>>>>> no clear consensus for or against backporting, but it > ended > > > up > > > > > not > > > > > > >>>>>>> happening. At the time, users could work around the issue > > by > > > > > > >>> enabling > > > > > > >>>>> the > > > > > > >>>>>>> Security Manager again via a command-line flag. > > > > > > >>>>>>> > > > > > > >>>>>>> Java 24, which is planned to release tomorrow, no longer > > has > > > > this > > > > > > >>>>>>> workaround available. > > > > > > >>>>>>> > > > > > > >>>>>>> This leaves users running Java 23 (I am one) in a > slightly > > > > > > >>>>> uncomfortable > > > > > > >>>>>>> spot. > > > > > > >>>>>>> > > > > > > >>>>>>> If Kafka releases 4.0 in the next month, we can rush to > > > upgrade > > > > > to > > > > > > >>>>> that, > > > > > > >>>>>>> and hope that the first release has no regressions. > > > > > > >>>>>>> > > > > > > >>>>>>> Otherwise, we will need to downgrade back to Java 21, > since > > > > > staying > > > > > > >>>> on > > > > > > >>>>> 23 > > > > > > >>>>>>> isn't a good idea past Oracle's quarterly security update > > in > > > > > April > > > > > > >>>> (see > > > > > > >>>>>>> https://www.oracle.com/security-alerts/), which will > > include > > > > > > >>> patches > > > > > > >>>>>> that > > > > > > >>>>>>> won't be released for Java 23. > > > > > > >>>>>>> > > > > > > >>>>>>> Would there be strong objections to attempting a backport > > of > > > > this > > > > > > >>>> shim > > > > > > >>>>>> to a > > > > > > >>>>>>> 3.9.x release? > > > > > > >>>>>>> > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>>> > > > > > > >>> > > > > > > > > > > > > > > > > > > > > >
