FYI: I opened https://github.com/apache/grails-core/issues/15807 for this. I'll open an upstream gradle ticket on this as well. For now, the 8.0.0-M2 is restaging and hopefully that fixes the last reproducibility issue.
On Wed, Jul 1, 2026 at 1:37 PM James Daugherty <[email protected]> wrote: > > I found the issue; it's a Gradle bug. I fed the information to AI and > had it reproduce it. Below is basically a bug report. I'll be fixing > in the code line & then opening a gradle ticket eventually: > > # Forked Groovy compiler daemon reuse leaks `jvmArgumentProviders` > system properties between compile tasks > > ## Summary > > When multiple `GroovyCompile` tasks run with `options.fork = true`, > Gradle reuses a single > forked compiler daemon (JVM) across tasks. JVM system properties contributed > via > `options.forkOptions.jvmArgumentProviders` (a > `CommandLineArgumentProvider`) are applied > **once, at daemon start**, and are **not** part of the key Gradle uses > to decide whether a > daemon can be reused for another task. > > The result: a compiler daemon first started for task **A** (which sets > `-Dfoo=A` via a > `jvmArgumentProvider`) is later reused to run task **B** (which sets a > *different* value, or > none). Inside task B's compilation, `System.getProperty("foo")` still > returns **A's** value. > Any AST transformation / compiler plugin that reads a system property > therefore sees a value > belonging to a *different* project. > > This is a correctness issue: a task's requested fork JVM arguments are > silently ignored when a > daemon is reused, and one task's process-global state bleeds into another. > > ## Environment > > - Gradle: **9.6.0** > - JDK: Liberica **21.0.7** (also reproduced on 21.x generally) > - Groovy: 4.0.x (`groovy` plugin, `GroovyCompile`) > - OS: reproduced on macOS (local) and Ubuntu (GitHub Actions CI) > - Relevant setting: `org.gradle.parallel=true`; also reproduces with > `--max-workers=1` > > ## Expected behavior > > For two `GroovyCompile` tasks whose forked JVMs are configured with > *different* JVM arguments > (whether via `forkOptions.jvmArgs` **or** > `forkOptions.jvmArgumentProviders`), one of the > following should hold: > > 1. The daemon started for task A is **not** reused for task B (the > differing JVM args make the > daemons incompatible), **or** > 2. The provider-contributed arguments are re-applied per task. > > In other words, `jvmArgumentProviders` should participate in > compiler-daemon reuse decisions > the same way `jvmArgs` do, so a task never runs in a daemon configured > with another task's > JVM arguments. > > ## Actual behavior > > `jvmArgumentProviders` are applied to the daemon JVM at startup but > are **excluded from the > daemon-reuse key**. A daemon is reused across tasks with different > provider-contributed system > properties, and the daemon retains the system properties of whichever > task first started it. > Subsequent tasks observe the wrong (stale) system-property values at > compile time. > > There is an inconsistency between the two ways of setting fork JVM args: > - `forkOptions.jvmArgs = ['-Dfoo=A']` — affects daemon identity > (partitions daemons). > - `forkOptions.jvmArgumentProviders.add { ['-Dfoo=A'] }` — applied to > the JVM, but does **not** > affect daemon identity (daemons are shared across differing values). > > ## Real-world impact (how we found it) > > We hit this in the Apache Grails build (a ~200-module Gradle project). > Grails contributes a > `-Dbase.dir=<projectDir>` system property to the Groovy compiler fork via a > `CommandLineArgumentProvider` (so it is configuration-cache safe). A > compile-time global AST > transformation reads `System.getProperty("base.dir")` to locate the > module being compiled and > merge that module's checked-in `META-INF/grails.factories` into the > generated output. > > Because the compiler daemon is reused, modules that did **not** set > `base.dir` (or set a > different one) read the `base.dir` of an unrelated module and merged > **that** module's > `grails.factories` into their own output — producing published > artifacts that contain > service-registration entries pointing at classes from a different > module (and not present in > the jar). This is non-deterministic: which module "wins" depends on > daemon scheduling, so the > same source produced different artifacts on different machines/CI > runs, breaking reproducible > builds. > > ### Direct evidence > > We instrumented the transform to log the compiler-daemon PID, the > compilation target > directory, and `System.getProperty("base.dir")` for every compile, then ran > `./gradlew assemble --max-workers=1 --rerun-tasks --no-build-cache` (with > `org.gradle.parallel=true`). Representative output (one line per > compiled module): > > ``` > pid=68685 compiled=grails-cache saw base.dir=grails-cache > <- daemon started here > pid=68685 compiled=grails-converters saw base.dir=grails-cache > <- REUSED, stale value > pid=68685 compiled=grails-rest-transforms saw base.dir=grails-cache > <- REUSED, stale value > pid=68813 compiled=grails-plugin saw base.dir=grails-plugin > <- daemon started here > pid=68813 compiled=grails-interceptors saw base.dir=grails-plugin > <- REUSED, stale value > pid=68571 compiled=grails-controllers saw base.dir=null > <- daemon started with no base.dir > pid=68571 compiled=grails-domain-class saw base.dir=null > <- REUSED (also fine) > ``` > > A single daemon PID compiles several modules and reports a **fixed** > `base.dir` — the value > belonging to whichever module first started that daemon — regardless > of what the currently > compiling module configured. > > ## Minimal reproduction > > A standalone project reproducing the leak without any of the Grails > machinery. Three pieces: > a global AST transform that records the system property it observes at > compile time, and two > modules whose compiler forks request *different* values via > `jvmArgumentProviders`. > > ### `settings.gradle` > ```groovy > rootProject.name = 'daemon-fork-arg-leak' > include 'transform', 'moduleA', 'moduleB' > ``` > > ### `transform/build.gradle` > ```groovy > plugins { id 'groovy' } > repositories { mavenCentral() } > dependencies { implementation "org.apache.groovy:groovy:4.0.28" } > ``` > > ### `transform/src/main/groovy/repro/ProbeTransform.groovy` > ```groovy > package repro > > import org.codehaus.groovy.ast.ASTNode > import org.codehaus.groovy.control.CompilePhase > import org.codehaus.groovy.control.SourceUnit > import org.codehaus.groovy.transform.ASTTransformation > import org.codehaus.groovy.transform.GroovyASTTransformation > > @GroovyASTTransformation(phase = CompilePhase.CANONICALIZATION) > class ProbeTransform implements ASTTransformation { > @Override > void visit(ASTNode[] nodes, SourceUnit source) { > def log = new File(System.getProperty('repro.log')) > log.append("pid=${ProcessHandle.current().pid()} " + > "compiling=${source.name} " + > "repro.base=${System.getProperty('repro.base')}\n") > } > } > ``` > > ### > `transform/src/main/resources/META-INF/services/org.codehaus.groovy.transform.ASTTransformation` > ``` > repro.ProbeTransform > ``` > > ### `moduleA/build.gradle` > ```groovy > plugins { id 'groovy' } > repositories { mavenCentral() } > dependencies { > implementation "org.apache.groovy:groovy:4.0.28" > implementation project(':transform') // puts the global > transform on the compile classpath > } > def reproLog = > rootProject.layout.buildDirectory.file('probe.log').get().asFile.absolutePath > tasks.withType(GroovyCompile).configureEach { > options.fork = true > // A sets repro.base=A via a CommandLineArgumentProvider > options.forkOptions.jvmArgumentProviders.add({ ['-Drepro.base=A', > "-Drepro.log=${reproLog}".toString()] } as > CommandLineArgumentProvider) > } > ``` > > ### `moduleB/build.gradle` > ```groovy > plugins { id 'groovy' } > repositories { mavenCentral() } > dependencies { > implementation "org.apache.groovy:groovy:4.0.28" > implementation project(':transform') > } > def reproLog = > rootProject.layout.buildDirectory.file('probe.log').get().asFile.absolutePath > tasks.withType(GroovyCompile).configureEach { > options.fork = true > // B sets repro.base=B (a DIFFERENT value) via a > CommandLineArgumentProvider > options.forkOptions.jvmArgumentProviders.add({ ['-Drepro.base=B', > "-Drepro.log=${reproLog}".toString()] } as > CommandLineArgumentProvider) > } > ``` > > ### One source file per module > ```groovy > // moduleA/src/main/groovy/a/A.groovy > package a > class A {} > ``` > ```groovy > // moduleB/src/main/groovy/b/B.groovy > package b > class B {} > ``` > > ### Run > ```bash > ./gradlew :moduleA:compileGroovy :moduleB:compileGroovy > --max-workers=1 --rerun-tasks > cat build/probe.log > ``` > > ### Expected `build/probe.log` > ``` > pid=<X> compiling=.../A.groovy repro.base=A > pid=<Y> compiling=.../B.groovy repro.base=B > ``` > > ### Actual `build/probe.log` (bug) > ``` > pid=<X> compiling=.../A.groovy repro.base=A > pid=<X> compiling=.../B.groovy repro.base=A <-- same daemon PID, B > sees A's value > ``` > > > Note: to make the daemon reuse deterministic, run the two compile tasks in > > a single > > invocation. Whether B observes `A` or `B` depends only on which task > > started the shared > > daemon; the point is that B runs in a daemon configured with A's JVM > > arguments. > > ## Suggested fix / discussion > > The core inconsistency is that `forkOptions.jvmArgs` participate in > compiler-daemon reuse > identity but `forkOptions.jvmArgumentProviders` do not, even though > both end up on the forked > JVM's command line. Resolving the providers and including their output in the > `DaemonForkOptions` fingerprint (used for reuse) would make daemon > reuse correct: a daemon > would only be reused for a task whose fully-resolved fork JVM arguments match. > > If that is intended behavior (providers deliberately excluded from > daemon identity), it would > help to document it prominently, because it makes > `jvmArgumentProviders` unsafe for passing any > per-task system property when compiler daemons are shared — a subtle, > data-dependent > correctness trap. > > ## Workarounds we are evaluating (for reference) > > - Passing per-task data as static `forkOptions.jvmArgs` instead of via > a provider (partitions > daemons, but is not configuration-cache friendly with absolute > paths, and disables daemon > reuse across modules). > - Not relying on a process-global system property to identify the > module being compiled; > deriving it from the per-compilation > `CompilerConfiguration.targetDirectory` instead. > > On Tue, Jun 30, 2026 at 7:49 PM James Daugherty <[email protected]> wrote: > > > > I can't seem to reproduce this issue locally. I've opened > > https://github.com/apache/grails-core/pull/15799 as a guard for this issue > > so we can fix once we know what's causing it. If I can get 2 reviewers, I > > can reroll the release. > > > > On 2026/06/30 22:02:04 James Daugherty wrote: > > > Just an update: I've located 2 issues with the build after fixing the > > > licensing issue. > > > > > > 1. there are build reproducibility issues that appear related to the > > > Gradle 9 upgrade. > > > 2. the micronaut island logic was recompiling everything under java 25, > > > which meant the binaries from the script were v25 while the binaries in > > > github action build were v21. > > > > > > I've fixed #2, and am researching #1. I'll follow through once I figure > > > out a solution. > > > > > > -James > > > > > > On 2026/06/30 16:09:13 James Daugherty wrote: > > > > FYI: GitHub is having issues running the abort action. I'm seeing a > > > > 500 error. I intend to restage this later today or tomorrow as a > > > > result. > > > > > > > > On 2026/06/30 14:54:32 James Daugherty wrote: > > > > > The initial vote for 8.0.0-M2 > > > > > (https://lists.apache.org/thread/ffqocxwqo7bkmlnjpgq5nntobmbcnbrw) is > > > > > cancelled due to a licensing issue with the build. I'll reroll this > > > > > today and try to get a new vote sent this afternoon. > > > > > > > > > > -James > > > > > > > > > > > >
