I have a draft PR up at https://github.com/apache/beam/pull/34915 with some changes and I'm cleaning up some remaining issues. I merged the javaVersion and testJavaVersion properties since the compilation toolchain shouldn't dictate binary compatibility with the minimum supported runtime version. Instead, minimumLanguageVersion on the Java nature dictates binary compatibility for the current toolchain (javac -release flag). In a few cases the selected toolchain for test execution is explicitly overridden in a build file (e.g. for Samza, Spark and Hadoop tests).
All of that allowed me to clean up the GitHub Actions configurations a bit, removing overrides of javaVersion from workflow configurations as much as possible. The build toolchain for all workflows uses JDK 11 for compilation targeting Java 8 binary compatibility by default and Gradle automatically invokes JRE 8 when required. Cheers, Steve On Tue, May 6, 2025 at 7:40 PM Yi Hu <ya...@google.com> wrote: > Thanks for raising this topic. > > Toolchain sounds a promising solution for challenges that we currently > have, e.g. in https://github.com/apache/beam/issues/34747. I've been > working on a home-made solution https://github.com/apache/beam/pull/34858. > I wonder using java toolchain could be much simpler. So does > test.javaLauncher that could replace the home-made `-PtestJavaVersion` > > Yi > > > > > On Tue, May 6, 2025 at 1:26 PM Kenneth Knowles <k...@apache.org> wrote: > >> We've actually been stuck since at least 2022 AFAIK ( >> https://github.com/apache/beam/pull/24245 and I think there have been >> other attempts as people discover it is not possible) >> >> I'm generally for moves that unblock Beam from using up-to-date tools and >> techniques, as long as we support our users who are on older versions. >> >> From what I can tell, options.release is actually more robust, as >> sourceCompatibility + targetCompatibility does not ensure that you compile >> against that version's inbuilt libraries. I would probably defer to you and >> @Yi >> Hu <ya...@google.com> and other members of the community who have more >> experience with these variations. >> >> Kenn >> >> On Tue, May 6, 2025 at 1:14 PM Steven van Rossum via dev < >> dev@beam.apache.org> wrote: >> >>> Hi all, >>> >>> I was hoping to update Error Prone to version 2.38.0 so we can turn on >>> the ThreadSafe checker (https://errorprone.info/bugpattern/ThreadSafe), >>> but it looks like that checker might not work just yet ( >>> https://github.com/google/error-prone/issues/4833). >>> >>> While preparing for that change I found out that recent versions of >>> Error Prone minimally require JDK 17 for compilation. It occurred to me >>> that Beam's toolchain management could be improved with Gradle's builtin >>> toolchain management ( >>> https://docs.gradle.org/current/userguide/toolchains.html). >>> Gradle's toolchain management combined with toolchain auto-discovery >>> make it a bit easier to run builds without having to explicitly override >>> JAVA_HOME, org.gradle.java.home or any of the Beam properties for selecting >>> runtimes (java8Home, java11Home, testJavaVersion). >>> >>> A brief overview of some of the changes: >>> - Replace java{VERSION}Home properties >>> with org.gradle.java.installations.paths=/path/to/java8,/path/to/java11,... >>> and enable/disable toolchain auto-discovery with >>> org.gradle.java.installations.auto-detect. >>> - Reconfigure (Java) compile tasks to use options.release instead of >>> options.sourceCompatibility and options.targetCompatibility to keep >>> producing JVM 8 compatible binaries. >>> - Reconfigure release specific test tasks to use test.javaLauncher for >>> JVM selection. >>> >>> Any thoughts on this? >>> >>> Cheers, >>> >>> Steve >>> >>
