Hi Stig, Thanks for the investigation. I'll take a look.
Thanks. Luke On Fri, Apr 4, 2025 at 2:24 AM Stig Rohde Døssing <stigdoess...@gmail.com> wrote: > The person doing the PR to update the Jenkinsfile should include these > changes, which were in the PR for trunk that updated Github Actions to run > Java 23 > > > https://github.com/apache/kafka/commit/76a9df47ca6088361fe707a2946827f7558d9f71 > > Den tors. 3. apr. 2025 kl. 20.17 skrev Stig Rohde Døssing < > stigdoess...@gmail.com>: > > > Looking in the build log, I see this > > > > Loading trusted files from base branch 3.9 at > > 90ee2d2b3499b7e1eff99bc63bf66aba9d7ff99c rather than > > f4cd8d5d664a62b280c0d2ea1bc731069443ba87 > > Obtained Jenkinsfile from 90ee2d2b3499b7e1eff99bc63bf66aba9d7ff99c > > ‘Jenkinsfile’ has been modified in an untrusted revision > > > > Going by an older blogpost at > > https://www.jenkins.io/blog/2022/12/14/require-java-11/, this means that > > Jenkins will only test Jenkinsfile changes in PRs submitted by people > with > > repo write access. > > > > I've run the tests for the https://github.com/apache/kafka/pull/19311 > > branch locally, and they pass. So maybe the easiest path forward is to > > review (and ideally merge) the following PRs > > > > https://github.com/apache/kafka/pull/19307 > > https://github.com/apache/kafka/pull/19308 > > https://github.com/apache/kafka/pull/19309 > > https://github.com/apache/kafka/pull/19310 > > https://github.com/apache/kafka/pull/19221 > > https://github.com/apache/kafka/pull/19365 > > https://github.com/apache/kafka/pull/19367 (note that this upgrades to > > the same Mockito version as used on trunk, while the WIP PR used the > latest > > Mockito. I don't think it should make a difference, but it seemed weird > to > > let 3.9 get ahead of the version trunk uses) > > > > which are the component parts of that branch, except for the Jenkinsfile > > update. > > > > Once these are merged, a committer can do a PR to update the Jenkinsfile > > to build on Java 23 as well. > > > > Den ons. 2. apr. 2025 kl. 17.06 skrev Stig Rohde Døssing < > > stigdoess...@gmail.com>: > > > >> Hi Luke, > >> > >> I took a look at getting 3.9 to run Java 23 on CI in this PR > >> https://github.com/apache/kafka/pull/19311 which is just 3.9 plus all > >> the other backport PRs, plus a few extra backports and changes to get > Java > >> 23 to build locally. I was planning to raise PRs for these bits > >> individually once I had a build working. > >> > >> But I am roadblocked on getting Jenkins to actually use Java 23. The CI > >> run appears to not have picked up the change (see > >> > https://github.com/apache/kafka/pull/19311/files#diff-e6ffa5dc854b843b3ee3c3c28f8eae2f436c2df2b1ca299cca1fa5982e390cf8 > ) > >> to the script I did in the last commit. I'm not familiar enough with > 3.9's > >> CI setup to know how to fix it, and I also don't think I have the > necessary > >> access to Jenkins to go dig into the job configuration to figure it out. > >> > >> In particular I don't know if > >> > >> tools { > >> jdk 'jdk_23_latest' > >> } > >> > >> is right, because I don't know which JDKs are available on Jenkins, and > >> without admin access, I don't believe I can get a list anywhere. > >> > >> If someone has full access to Jenkins, maybe they can take a look? > >> > >> Den ons. 2. apr. 2025 kl. 14.04 skrev Dejan Stojadinović < > >> dejan2...@gmail.com>: > >> > >>> Hi all, > >>> > >>> Just keep in mind that *Gradle version 8.14 *(currently under > >>> development) > >>> will support *Java 24*: > >>> > >>> - https://docs.gradle.org/8.13/userguide/compatibility.html > >>> - > >>> https://github.com/gradle/gradle/issues/32290#issuecomment-2741096604 > >>> - > >>> https://github.com/gradle/gradle/issues/32289#issuecomment-2741056207 > >>> > >>> > >>> > >>> On Wed, Apr 2, 2025 at 1:25 PM Luke Chen <show...@gmail.com> wrote: > >>> > >>> > Hi Stig and all, > >>> > > >>> > For backporting the change to 3.9, if there's no objections, I think > >>> we can > >>> > start to merge them tomorrow. > >>> > For the Java 24 testing, it's fine we ship 3.9 without Java 24 test > if > >>> > gradle release is delayed since this is also the state of what Kafka > >>> v4.0 > >>> > is. > >>> > > >>> > So we should firstly test Java 23 in CI for 3.9. @Stig, could you > help > >>> open > >>> > a PR for it? > >>> > > >>> > Thanks. > >>> > Luke > >>> > > >>> > On Sat, Mar 29, 2025 at 2:29 AM Stig Rohde Døssing < > >>> stigdoess...@gmail.com > >>> > > > >>> > wrote: > >>> > > >>> > > Thanks. I'll wait for that PR then. Since it's blocked by Gradle, > >>> and the > >>> > > plan is for Gradle to release a new version on April 7th (see > >>> > > > >>> https://github.com/gradle/gradle/issues/32289#issuecomment-2741056207 > ), > >>> > > maybe the way to go is to aim to test with Java 23 on 3.9 for now? > >>> Once > >>> > we > >>> > > can test with Java 24 on trunk, maybe we can consider backporting > the > >>> > > necessary changes to 3.9 to get that branch tested against Java 24 > as > >>> > well, > >>> > > and getting it running with Java 23 at least gets us closer. > >>> > > > >>> > > Regarding CI, I made an attempt at getting Jenkins to run with Java > >>> 23 > >>> > here > >>> > > > >>> > > > >>> > > >>> > https://github.com/apache/kafka/pull/19311/commits/f4cd8d5d664a62b280c0d2ea1bc731069443ba87 > >>> > > , > >>> > > but it doesn't look like CI is picking up the change. Maybe Jenkins > >>> is > >>> > > configured to fetch the script from a specific branch? I can't tell > >>> > since I > >>> > > don't have access to the job configurations. > >>> > > > >>> > > Den fre. 28. mar. 2025 kl. 19.00 skrev Chia-Ping Tsai < > >>> > chia7...@gmail.com > >>> > > >: > >>> > > > >>> > > > hi Stig > >>> > > > > >>> > > > > Does anyone know where the corresponding CI script > >>> > > > files are located in 3.9? > >>> > > > > >>> > > > https://github.com/apache/kafka/blob/3.9/Jenkinsfile#L165 > >>> > > > > >>> > > > > I noticed that trunk isn't currently testing against Java 24. > >>> I'll > >>> > try > >>> > > to > >>> > > > make a PR to fix that, which will probably require another bump > of > >>> > > > Spotbugs, which may prompt another backport PR. > >>> > > > > >>> > > > there is already a PR ( > https://github.com/apache/kafka/pull/19235) > >>> > > > > >>> > > > Best, > >>> > > > Chia-Ping > >>> > > > > >>> > > > Stig Rohde Døssing <stigdoess...@gmail.com> 於 2025年3月29日 週六 > >>> 上午12:02寫道： > >>> > > > > >>> > > > > Chia-Ping, I think we should aim for Java 24 support, since 23 > >>> > reaches > >>> > > > end > >>> > > > > of life in a month or so. > >>> > > > > > >>> > > > > Here are PRs for 4 of the changes linked at > >>> > > > > https://issues.apache.org/jira/browse/KAFKA-17638 > >>> > > > > > >>> > > > > Scala upgrade https://github.com/apache/kafka/pull/19307 > >>> > > > > Compiler warning 1 https://github.com/apache/kafka/pull/19308 > >>> > > > > Compiler warning 2 https://github.com/apache/kafka/pull/19309 > >>> > > > > Spotbugs update https://github.com/apache/kafka/pull/19310 > >>> > > > > > >>> > > > > The missing backports after that will be > >>> > > > > https://github.com/apache/kafka/pull/17409 and > >>> > > > > https://github.com/apache/kafka/pull/17403 > >>> > > > > > >>> > > > > For the former PR, it cherry picks mostly cleanly, but it won't > >>> pass > >>> > > > tests > >>> > > > > until the other changes are merged. The CI script changes are > >>> done to > >>> > > > files > >>> > > > > that don't exist in 3.9. Does anyone know where the > >>> corresponding CI > >>> > > > script > >>> > > > > files are located in 3.9? > >>> > > > > > >>> > > > > The latter PR contains only documentation changes calling out > >>> that > >>> > Java > >>> > > > 23 > >>> > > > > is now supported. I think we are better off adjusting the docs > on > >>> > trunk > >>> > > > to > >>> > > > > say that Java 24 is supported as of 3.9.1 instead of > backporting > >>> that > >>> > > > > change. > >>> > > > > > >>> > > > > I noticed that trunk isn't currently testing against Java 24. > >>> I'll > >>> > try > >>> > > to > >>> > > > > make a PR to fix that, which will probably require another bump > >>> of > >>> > > > > Spotbugs, which may prompt another backport PR. > >>> > > > > > >>> > > > > Den fre. 28. mar. 2025 kl. 15.08 skrev Chia-Ping Tsai < > >>> > > > chia7...@gmail.com > >>> > > > > >: > >>> > > > > > >>> > > > > > hi Stig > >>> > > > > > > >>> > > > > > We should run CI under Java 23 if we want to make 3.9 support > >>> Java > >>> > 23 > >>> > > > > > officially, so +1 to backport them > >>> > > > > > > >>> > > > > > Best, > >>> > > > > > Chia-Ping > >>> > > > > > > >>> > > > > > > >>> > > > > > > Mickael Maison <mickael.mai...@gmail.com> 於 2025年3月28日 > >>> 下午6:43 > >>> > 寫道： > >>> > > > > > > > >>> > > > > > > Hi, > >>> > > > > > > > >>> > > > > > > I'm also supportive of backporting this change to the 3.9 > >>> branch. > >>> > > > > > > > >>> > > > > > > Thanks, > >>> > > > > > > Mickael > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > >> On Fri, Mar 28, 2025 at 10:16 AM Stig Rohde Døssing > >>> > > > > > >> <stigdoess...@gmail.com> wrote: > >>> > > > > > >> > >>> > > > > > >> Pulling in a discussion from the PR thread: > >>> > > > > > >> > >>> > > > > > >>> There were other PRs that were part of Java 23 support > >>> > generally, > >>> > > > > > should > >>> > > > > > >> these also be backported? > >>> > > > > > https://issues.apache.org/jira/browse/KAFKA-17638 > >>> > > > > > >> > >>> > > > > > >> Greg, thanks for letting me know about these, I was not > >>> aware. > >>> > > > > > >> > >>> > > > > > >> I think most likely these changes aren't necessary to > allow > >>> > Kafka > >>> > > to > >>> > > > > > run on > >>> > > > > > >> Java 23. I believe upgrading Scala is necessary because > the > >>> > > bundled > >>> > > > > ASM > >>> > > > > > >> doesn't understand Java 23 bytecode, and the remaining > >>> changes > >>> > > seem > >>> > > > to > >>> > > > > > >> relate to either fixing compiler warnings or adding Java > 23 > >>> to > >>> > the > >>> > > > > build > >>> > > > > > >> matrix. I think backporting these would be a fine idea, > but > >>> I > >>> > > don't > >>> > > > > > think > >>> > > > > > >> they're blockers for using Kafka on Java 23 or 24. > >>> > > > > > >> > >>> > > > > > >> I'm happy to open PRs to backport these as well (though > for > >>> the > >>> > > > build > >>> > > > > > >> matrix, we should use Java 24 now), assuming there's no > >>> strong > >>> > > > > > opposition > >>> > > > > > >> to that? > >>> > > > > > >> > >>> > > > > > >> Den fre. 28. mar. 2025 kl. 05.02 skrev Greg Harris > >>> > > > > > >> <greg.har...@aiven.io.invalid>: > >>> > > > > > >> > >>> > > > > > >>> Hi All, > >>> > > > > > >>> > >>> > > > > > >>> I am still strongly in support of backporting this patch > >>> as I > >>> > > > stated > >>> > > > > > in the > >>> > > > > > >>> earlier discussion thread. > >>> > > > > > >>> > >>> > > > > > >>> Thank you Stig, Anton, Danish, Mateusz, Manfred, Monica, > >>> > Istvan, > >>> > > > > > Vincent, > >>> > > > > > >>> Clement, Anshu, Foivos, Drakgoku, Severin, Ozan, > Georgios, > >>> and > >>> > > > > > Guillaume, > >>> > > > > > >>> members of our extended community who have commented to > >>> discuss > >>> > > the > >>> > > > > > >>> impact and their support for this patch. > >>> > > > > > >>> I inevitably missed some people, such as everyone who has > >>> been > >>> > > > > > encountering > >>> > > > > > >>> and silently working around the problem just from reading > >>> these > >>> > > > > > threads and > >>> > > > > > >>> GitHub issues. > >>> > > > > > >>> > >>> > > > > > >>> Through our inaction, we have obligated several users to > >>> take > >>> > > > actions > >>> > > > > > to > >>> > > > > > >>> work around us: > >>> > > > > > >>> * The Trino project dropped two of their products [1, 2] > >>> > > > > > >>> * The Quarkus project disabled some tests [3] > >>> > > > > > >>> * Quarkus project users downgraded their java version [4] > >>> > > > > > >>> * Spring Boot users have installed the system property > >>> > workaround > >>> > > > [5] > >>> > > > > > >>> > >>> > > > > > >>> In my opinion this is unacceptable, and it's time for us > >>> to fix > >>> > > > this. > >>> > > > > > >>> > >>> > > > > > >>> Thank you Stig for restarting the conversation! > >>> > > > > > >>> Greg > >>> > > > > > >>> > >>> > > > > > >>> [1] https://github.com/trinodb/trino/issues/24419 > >>> > > > > > >>> [2] https://github.com/trinodb/trino/issues/24417 > >>> > > > > > >>> [3] https://github.com/quarkusio/quarkus/pull/43543 > >>> > > > > > >>> [4] > >>> > > > > > >>> > >>> > > > > > >>> > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > https://github.com/anshupitlia/product-information-system/commit/2c3a8dbd974dce0273f74969ec64b661abafef62 > >>> > > > > > >>> [5] > >>> > > > > > >>> > >>> > > > > > >>> > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > https://github.com/vividus-framework/vividus-build-system/commit/53b7016a0d3b0ba04a23b4b1892e1cf7f62ba0a5 > >>> > > > > > >>> > >>> > > > > > >>>> On Thu, Mar 27, 2025 at 7:39 PM Luke Chen < > >>> show...@gmail.com> > >>> > > > > wrote: > >>> > > > > > >>> > >>> > > > > > >>>> Hi Stig, > >>> > > > > > >>>> > >>> > > > > > >>>> Thanks for bringing this to us. > >>> > > > > > >>>> I'm +1 for backporting to 3.9 branch since there's no > >>> > workaround > >>> > > > for > >>> > > > > > Java > >>> > > > > > >>>> 24. > >>> > > > > > >>>> > >>> > > > > > >>>> Thanks. > >>> > > > > > >>>> Luke > >>> > > > > > >>>> > >>> > > > > > >>>> > >>> > > > > > >>>> > >>> > > > > > >>>> On Tue, Mar 18, 2025 at 1:14 AM Stig Rohde Døssing < > >>> > > > > > >>> stigdoess...@gmail.com > >>> > > > > > >>>>> > >>> > > > > > >>>> wrote: > >>> > > > > > >>>> > >>> > > > > > >>>>> Thanks Ismail, > >>> > > > > > >>>>> > >>> > > > > > >>>>> I've opened https://github.com/apache/kafka/pull/19221 > >>> just > >>> > to > >>> > > > get > >>> > > > > > any > >>> > > > > > >>>>> test > >>> > > > > > >>>>> failures out of the way in case it is decided to do > this > >>> > > > backport. > >>> > > > > > >>>>> > >>> > > > > > >>>>> I'm hoping people will weigh in with their concerns in > >>> this > >>> > > > thread > >>> > > > > if > >>> > > > > > >>>> they > >>> > > > > > >>>>> don't like the idea of backporting this change. > >>> > > > > > >>>>> > >>> > > > > > >>>>> Den man. 17. mar. 2025 kl. 16.43 skrev Ismael Juma < > >>> > > > > > m...@ismaeljuma.com > >>> > > > > > >>>> : > >>> > > > > > >>>>> > >>> > > > > > >>>>>> Hi Stig, > >>> > > > > > >>>>>> > >>> > > > > > >>>>>> Kafka 4.0 is likely to be released in a day or two. > Even > >>> > so, I > >>> > > > > think > >>> > > > > > >>> it > >>> > > > > > >>>>>> makes sense to revive the backporting thread given the > >>> lack > >>> > of > >>> > > > > > >>>> workaround > >>> > > > > > >>>>>> for Java 24. > >>> > > > > > >>>>>> > >>> > > > > > >>>>>> Ismael > >>> > > > > > >>>>>> > >>> > > > > > >>>>>> On Mon, Mar 17, 2025 at 7:44 AM Stig Rohde Døssing < > >>> > > > > > >>>>> stigdoess...@gmail.com > >>> > > > > > >>>>>>> > >>> > > > > > >>>>>> wrote: > >>> > > > > > >>>>>> > >>> > > > > > >>>>>>> Hi, > >>> > > > > > >>>>>>> > >>> > > > > > >>>>>>> Some months ago, a reflective shim was added in > >>> > > > > > >>>>>>> https://issues.apache.org/jira/browse/KAFKA-17078, > in > >>> > order > >>> > > to > >>> > > > > > >>>> support > >>> > > > > > >>>>>>> running Kafka with SASL on JDKs that no longer > support > >>> the > >>> > > > > security > >>> > > > > > >>>>>>> manager. > >>> > > > > > >>>>>>> > >>> > > > > > >>>>>>> This shim was added only to Kafka 4.0, but > backporting > >>> was > >>> > > > > > >>> discussed > >>> > > > > > >>>> in > >>> > > > > > >>>>>>> > >>> > > > https://lists.apache.org/thread/vl43q9wqq4xs67xx61f0t0850y2b037o > >>> > > > > . > >>> > > > > > >>>>> There > >>> > > > > > >>>>>>> was > >>> > > > > > >>>>>>> no clear consensus for or against backporting, but it > >>> ended > >>> > > up > >>> > > > > not > >>> > > > > > >>>>>>> happening. At the time, users could work around the > >>> issue > >>> > by > >>> > > > > > >>> enabling > >>> > > > > > >>>>> the > >>> > > > > > >>>>>>> Security Manager again via a command-line flag. > >>> > > > > > >>>>>>> > >>> > > > > > >>>>>>> Java 24, which is planned to release tomorrow, no > >>> longer > >>> > has > >>> > > > this > >>> > > > > > >>>>>>> workaround available. > >>> > > > > > >>>>>>> > >>> > > > > > >>>>>>> This leaves users running Java 23 (I am one) in a > >>> slightly > >>> > > > > > >>>>> uncomfortable > >>> > > > > > >>>>>>> spot. > >>> > > > > > >>>>>>> > >>> > > > > > >>>>>>> If Kafka releases 4.0 in the next month, we can rush > to > >>> > > upgrade > >>> > > > > to > >>> > > > > > >>>>> that, > >>> > > > > > >>>>>>> and hope that the first release has no regressions. > >>> > > > > > >>>>>>> > >>> > > > > > >>>>>>> Otherwise, we will need to downgrade back to Java 21, > >>> since > >>> > > > > staying > >>> > > > > > >>>> on > >>> > > > > > >>>>> 23 > >>> > > > > > >>>>>>> isn't a good idea past Oracle's quarterly security > >>> update > >>> > in > >>> > > > > April > >>> > > > > > >>>> (see > >>> > > > > > >>>>>>> https://www.oracle.com/security-alerts/), which will > >>> > include > >>> > > > > > >>> patches > >>> > > > > > >>>>>> that > >>> > > > > > >>>>>>> won't be released for Java 23. > >>> > > > > > >>>>>>> > >>> > > > > > >>>>>>> Would there be strong objections to attempting a > >>> backport > >>> > of > >>> > > > this > >>> > > > > > >>>> shim > >>> > > > > > >>>>>> to a > >>> > > > > > >>>>>>> 3.9.x release? > >>> > > > > > >>>>>>> > >>> > > > > > >>>>>> > >>> > > > > > >>>>> > >>> > > > > > >>>> > >>> > > > > > >>> > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >> >
