I found the issue; it's a Gradle bug. I fed the information to AI and
had it reproduce it. Below is basically a bug report. I'll be fixing
in the code line & then opening a gradle ticket eventually:
# Forked Groovy compiler daemon reuse leaks `jvmArgumentProviders`
system properties between compile tasks
## Summary
When multiple `GroovyCompile` tasks run with `options.fork = true`,
Gradle reuses a single
forked compiler daemon (JVM) across tasks. JVM system properties contributed via
`options.forkOptions.jvmArgumentProviders` (a
`CommandLineArgumentProvider`) are applied
**once, at daemon start**, and are **not** part of the key Gradle uses
to decide whether a
daemon can be reused for another task.
The result: a compiler daemon first started for task **A** (which sets
`-Dfoo=A` via a
`jvmArgumentProvider`) is later reused to run task **B** (which sets a
*different* value, or
none). Inside task B's compilation, `System.getProperty("foo")` still
returns **A's** value.
Any AST transformation / compiler plugin that reads a system property
therefore sees a value
belonging to a *different* project.
This is a correctness issue: a task's requested fork JVM arguments are
silently ignored when a
daemon is reused, and one task's process-global state bleeds into another.
## Environment
- Gradle: **9.6.0**
- JDK: Liberica **21.0.7** (also reproduced on 21.x generally)
- Groovy: 4.0.x (`groovy` plugin, `GroovyCompile`)
- OS: reproduced on macOS (local) and Ubuntu (GitHub Actions CI)
- Relevant setting: `org.gradle.parallel=true`; also reproduces with
`--max-workers=1`
## Expected behavior
For two `GroovyCompile` tasks whose forked JVMs are configured with
*different* JVM arguments
(whether via `forkOptions.jvmArgs` **or**
`forkOptions.jvmArgumentProviders`), one of the
following should hold:
1. The daemon started for task A is **not** reused for task B (the
differing JVM args make the
daemons incompatible), **or**
2. The provider-contributed arguments are re-applied per task.
In other words, `jvmArgumentProviders` should participate in
compiler-daemon reuse decisions
the same way `jvmArgs` do, so a task never runs in a daemon configured
with another task's
JVM arguments.
## Actual behavior
`jvmArgumentProviders` are applied to the daemon JVM at startup but
are **excluded from the
daemon-reuse key**. A daemon is reused across tasks with different
provider-contributed system
properties, and the daemon retains the system properties of whichever
task first started it.
Subsequent tasks observe the wrong (stale) system-property values at
compile time.
There is an inconsistency between the two ways of setting fork JVM args:
- `forkOptions.jvmArgs = ['-Dfoo=A']` — affects daemon identity
(partitions daemons).
- `forkOptions.jvmArgumentProviders.add { ['-Dfoo=A'] }` — applied to
the JVM, but does **not**
affect daemon identity (daemons are shared across differing values).
## Real-world impact (how we found it)
We hit this in the Apache Grails build (a ~200-module Gradle project).
Grails contributes a
`-Dbase.dir=<projectDir>` system property to the Groovy compiler fork via a
`CommandLineArgumentProvider` (so it is configuration-cache safe). A
compile-time global AST
transformation reads `System.getProperty("base.dir")` to locate the
module being compiled and
merge that module's checked-in `META-INF/grails.factories` into the
generated output.
Because the compiler daemon is reused, modules that did **not** set
`base.dir` (or set a
different one) read the `base.dir` of an unrelated module and merged
**that** module's
`grails.factories` into their own output — producing published
artifacts that contain
service-registration entries pointing at classes from a different
module (and not present in
the jar). This is non-deterministic: which module "wins" depends on
daemon scheduling, so the
same source produced different artifacts on different machines/CI
runs, breaking reproducible
builds.
### Direct evidence
We instrumented the transform to log the compiler-daemon PID, the
compilation target
directory, and `System.getProperty("base.dir")` for every compile, then ran
`./gradlew assemble --max-workers=1 --rerun-tasks --no-build-cache` (with
`org.gradle.parallel=true`). Representative output (one line per
compiled module):
```
pid=68685 compiled=grails-cache saw base.dir=grails-cache
<- daemon started here
pid=68685 compiled=grails-converters saw base.dir=grails-cache
<- REUSED, stale value
pid=68685 compiled=grails-rest-transforms saw base.dir=grails-cache
<- REUSED, stale value
pid=68813 compiled=grails-plugin saw base.dir=grails-plugin
<- daemon started here
pid=68813 compiled=grails-interceptors saw base.dir=grails-plugin
<- REUSED, stale value
pid=68571 compiled=grails-controllers saw base.dir=null
<- daemon started with no base.dir
pid=68571 compiled=grails-domain-class saw base.dir=null
<- REUSED (also fine)
```
A single daemon PID compiles several modules and reports a **fixed**
`base.dir` — the value
belonging to whichever module first started that daemon — regardless
of what the currently
compiling module configured.
## Minimal reproduction
A standalone project reproducing the leak without any of the Grails
machinery. Three pieces:
a global AST transform that records the system property it observes at
compile time, and two
modules whose compiler forks request *different* values via
`jvmArgumentProviders`.
### `settings.gradle`
```groovy
rootProject.name = 'daemon-fork-arg-leak'
include 'transform', 'moduleA', 'moduleB'
```
### `transform/build.gradle`
```groovy
plugins { id 'groovy' }
repositories { mavenCentral() }
dependencies { implementation "org.apache.groovy:groovy:4.0.28" }
```
### `transform/src/main/groovy/repro/ProbeTransform.groovy`
```groovy
package repro
import org.codehaus.groovy.ast.ASTNode
import org.codehaus.groovy.control.CompilePhase
import org.codehaus.groovy.control.SourceUnit
import org.codehaus.groovy.transform.ASTTransformation
import org.codehaus.groovy.transform.GroovyASTTransformation
@GroovyASTTransformation(phase = CompilePhase.CANONICALIZATION)
class ProbeTransform implements ASTTransformation {
@Override
void visit(ASTNode[] nodes, SourceUnit source) {
def log = new File(System.getProperty('repro.log'))
log.append("pid=${ProcessHandle.current().pid()} " +
"compiling=${source.name} " +
"repro.base=${System.getProperty('repro.base')}\n")
}
}
```
###
`transform/src/main/resources/META-INF/services/org.codehaus.groovy.transform.ASTTransformation`
```
repro.ProbeTransform
```
### `moduleA/build.gradle`
```groovy
plugins { id 'groovy' }
repositories { mavenCentral() }
dependencies {
implementation "org.apache.groovy:groovy:4.0.28"
implementation project(':transform') // puts the global
transform on the compile classpath
}
def reproLog =
rootProject.layout.buildDirectory.file('probe.log').get().asFile.absolutePath
tasks.withType(GroovyCompile).configureEach {
options.fork = true
// A sets repro.base=A via a CommandLineArgumentProvider
options.forkOptions.jvmArgumentProviders.add({ ['-Drepro.base=A',
"-Drepro.log=${reproLog}".toString()] } as
CommandLineArgumentProvider)
}
```
### `moduleB/build.gradle`
```groovy
plugins { id 'groovy' }
repositories { mavenCentral() }
dependencies {
implementation "org.apache.groovy:groovy:4.0.28"
implementation project(':transform')
}
def reproLog =
rootProject.layout.buildDirectory.file('probe.log').get().asFile.absolutePath
tasks.withType(GroovyCompile).configureEach {
options.fork = true
// B sets repro.base=B (a DIFFERENT value) via a CommandLineArgumentProvider
options.forkOptions.jvmArgumentProviders.add({ ['-Drepro.base=B',
"-Drepro.log=${reproLog}".toString()] } as
CommandLineArgumentProvider)
}
```
### One source file per module
```groovy
// moduleA/src/main/groovy/a/A.groovy
package a
class A {}
```
```groovy
// moduleB/src/main/groovy/b/B.groovy
package b
class B {}
```
### Run
```bash
./gradlew :moduleA:compileGroovy :moduleB:compileGroovy
--max-workers=1 --rerun-tasks
cat build/probe.log
```
### Expected `build/probe.log`
```
pid=<X> compiling=.../A.groovy repro.base=A
pid=<Y> compiling=.../B.groovy repro.base=B
```
### Actual `build/probe.log` (bug)
```
pid=<X> compiling=.../A.groovy repro.base=A
pid=<X> compiling=.../B.groovy repro.base=A <-- same daemon PID, B
sees A's value
```
> Note: to make the daemon reuse deterministic, run the two compile tasks in a
> single
> invocation. Whether B observes `A` or `B` depends only on which task started
> the shared
> daemon; the point is that B runs in a daemon configured with A's JVM
> arguments.
## Suggested fix / discussion
The core inconsistency is that `forkOptions.jvmArgs` participate in
compiler-daemon reuse
identity but `forkOptions.jvmArgumentProviders` do not, even though
both end up on the forked
JVM's command line. Resolving the providers and including their output in the
`DaemonForkOptions` fingerprint (used for reuse) would make daemon
reuse correct: a daemon
would only be reused for a task whose fully-resolved fork JVM arguments match.
If that is intended behavior (providers deliberately excluded from
daemon identity), it would
help to document it prominently, because it makes
`jvmArgumentProviders` unsafe for passing any
per-task system property when compiler daemons are shared — a subtle,
data-dependent
correctness trap.
## Workarounds we are evaluating (for reference)
- Passing per-task data as static `forkOptions.jvmArgs` instead of via
a provider (partitions
daemons, but is not configuration-cache friendly with absolute
paths, and disables daemon
reuse across modules).
- Not relying on a process-global system property to identify the
module being compiled;
deriving it from the per-compilation
`CompilerConfiguration.targetDirectory` instead.
On Tue, Jun 30, 2026 at 7:49 PM James Daugherty <[email protected]> wrote:
>
> I can't seem to reproduce this issue locally. I've opened
> https://github.com/apache/grails-core/pull/15799 as a guard for this issue so
> we can fix once we know what's causing it. If I can get 2 reviewers, I can
> reroll the release.
>
> On 2026/06/30 22:02:04 James Daugherty wrote:
> > Just an update: I've located 2 issues with the build after fixing the
> > licensing issue.
> >
> > 1. there are build reproducibility issues that appear related to the Gradle
> > 9 upgrade.
> > 2. the micronaut island logic was recompiling everything under java 25,
> > which meant the binaries from the script were v25 while the binaries in
> > github action build were v21.
> >
> > I've fixed #2, and am researching #1. I'll follow through once I figure
> > out a solution.
> >
> > -James
> >
> > On 2026/06/30 16:09:13 James Daugherty wrote:
> > > FYI: GitHub is having issues running the abort action. I'm seeing a 500
> > > error. I intend to restage this later today or tomorrow as a result.
> > >
> > > On 2026/06/30 14:54:32 James Daugherty wrote:
> > > > The initial vote for 8.0.0-M2
> > > > (https://lists.apache.org/thread/ffqocxwqo7bkmlnjpgq5nntobmbcnbrw) is
> > > > cancelled due to a licensing issue with the build. I'll reroll this
> > > > today and try to get a new vote sent this afternoon.
> > > >
> > > > -James
> > > >
> > >
> >
