jdaugherty opened a new issue, #15807:
URL: https://github.com/apache/grails-core/issues/15807
### Issue description
# Forked Groovy compiler daemon reuse leaks `jvmArgumentProviders` system
properties between compile tasks
The GrailsAppBaseDirProvider thus causes the base.dir to be wrong in
projects that do not apply the grails gradle plugin.
## Summary
When multiple `GroovyCompile` tasks run with `options.fork = true`, Gradle
reuses a single
forked compiler daemon (JVM) across tasks. JVM system properties contributed
via
`options.forkOptions.jvmArgumentProviders` (a `CommandLineArgumentProvider`)
are applied
**once, at daemon start**, and are **not** part of the key Gradle uses to
decide whether a
daemon can be reused for another task.
The result: a compiler daemon first started for task **A** (which sets
`-Dfoo=A` via a
`jvmArgumentProvider`) is later reused to run task **B** (which sets a
*different* value, or
none). Inside task B's compilation, `System.getProperty("foo")` still
returns **A's** value.
Any AST transformation / compiler plugin that reads a system property
therefore sees a value
belonging to a *different* project.
This is a correctness issue: a task's requested fork JVM arguments are
silently ignored when a
daemon is reused, and one task's process-global state bleeds into another.
## Environment
- Gradle: **9.6.0**
- JDK: Liberica **21.0.7** (also reproduced on 21.x generally)
- Groovy: 4.0.x (`groovy` plugin, `GroovyCompile`)
- OS: reproduced on macOS (local) and Ubuntu (GitHub Actions CI)
- Relevant setting: `org.gradle.parallel=true`; also reproduces with
`--max-workers=1`
## Expected behavior
For two `GroovyCompile` tasks whose forked JVMs are configured with
*different* JVM arguments
(whether via `forkOptions.jvmArgs` **or**
`forkOptions.jvmArgumentProviders`), one of the
following should hold:
1. The daemon started for task A is **not** reused for task B (the differing
JVM args make the
daemons incompatible), **or**
2. The provider-contributed arguments are re-applied per task.
In other words, `jvmArgumentProviders` should participate in compiler-daemon
reuse decisions
the same way `jvmArgs` do, so a task never runs in a daemon configured with
another task's
JVM arguments.
## Actual behavior
`jvmArgumentProviders` are applied to the daemon JVM at startup but are
**excluded from the
daemon-reuse key**. A daemon is reused across tasks with different
provider-contributed system
properties, and the daemon retains the system properties of whichever task
first started it.
Subsequent tasks observe the wrong (stale) system-property values at compile
time.
There is an inconsistency between the two ways of setting fork JVM args:
- `forkOptions.jvmArgs = ['-Dfoo=A']` — affects daemon identity (partitions
daemons).
- `forkOptions.jvmArgumentProviders.add { ['-Dfoo=A'] }` — applied to the
JVM, but does **not**
affect daemon identity (daemons are shared across differing values).
## Real-world impact (how we found it)
We hit this in the Apache Grails build (a ~200-module Gradle project).
Grails contributes a
`-Dbase.dir=<projectDir>` system property to the Groovy compiler fork via a
`CommandLineArgumentProvider` (so it is configuration-cache safe). A
compile-time global AST
transformation reads `System.getProperty("base.dir")` to locate the module
being compiled and
merge that module's checked-in `META-INF/grails.factories` into the
generated output.
Because the compiler daemon is reused, modules that did **not** set
`base.dir` (or set a
different one) read the `base.dir` of an unrelated module and merged
**that** module's
`grails.factories` into their own output — producing published artifacts
that contain
service-registration entries pointing at classes from a different module
(and not present in
the jar). This is non-deterministic: which module "wins" depends on daemon
scheduling, so the
same source produced different artifacts on different machines/CI runs,
breaking reproducible
builds.
### Direct evidence
We instrumented the transform to log the compiler-daemon PID, the
compilation target
directory, and `System.getProperty("base.dir")` for every compile, then ran
`./gradlew assemble --max-workers=1 --rerun-tasks --no-build-cache` (with
`org.gradle.parallel=true`). Representative output (one line per compiled
module):
```
pid=68685 compiled=grails-cache saw base.dir=grails-cache <-
daemon started here
pid=68685 compiled=grails-converters saw base.dir=grails-cache <-
REUSED, stale value
pid=68685 compiled=grails-rest-transforms saw base.dir=grails-cache <-
REUSED, stale value
pid=68813 compiled=grails-plugin saw base.dir=grails-plugin <-
daemon started here
pid=68813 compiled=grails-interceptors saw base.dir=grails-plugin <-
REUSED, stale value
pid=68571 compiled=grails-controllers saw base.dir=null <-
daemon started with no base.dir
pid=68571 compiled=grails-domain-class saw base.dir=null <-
REUSED (also fine)
```
A single daemon PID compiles several modules and reports a **fixed**
`base.dir` — the value
belonging to whichever module first started that daemon — regardless of what
the currently
compiling module configured.
## Minimal reproduction
A standalone project reproducing the leak without any of the Grails
machinery. Three pieces:
a global AST transform that records the system property it observes at
compile time, and two
modules whose compiler forks request *different* values via
`jvmArgumentProviders`.
### `settings.gradle`
```groovy
rootProject.name = 'daemon-fork-arg-leak'
include 'transform', 'moduleA', 'moduleB'
```
### `transform/build.gradle`
```groovy
plugins { id 'groovy' }
repositories { mavenCentral() }
dependencies { implementation "org.apache.groovy:groovy:4.0.28" }
```
### `transform/src/main/groovy/repro/ProbeTransform.groovy`
```groovy
package repro
import org.codehaus.groovy.ast.ASTNode
import org.codehaus.groovy.control.CompilePhase
import org.codehaus.groovy.control.SourceUnit
import org.codehaus.groovy.transform.ASTTransformation
import org.codehaus.groovy.transform.GroovyASTTransformation
@GroovyASTTransformation(phase = CompilePhase.CANONICALIZATION)
class ProbeTransform implements ASTTransformation {
@Override
void visit(ASTNode[] nodes, SourceUnit source) {
def log = new File(System.getProperty('repro.log'))
log.append("pid=${ProcessHandle.current().pid()} " +
"compiling=${source.name} " +
"repro.base=${System.getProperty('repro.base')}\n")
}
}
```
###
`transform/src/main/resources/META-INF/services/org.codehaus.groovy.transform.ASTTransformation`
```
repro.ProbeTransform
```
### `moduleA/build.gradle`
```groovy
plugins { id 'groovy' }
repositories { mavenCentral() }
dependencies {
implementation "org.apache.groovy:groovy:4.0.28"
implementation project(':transform') // puts the global transform on
the compile classpath
}
def reproLog =
rootProject.layout.buildDirectory.file('probe.log').get().asFile.absolutePath
tasks.withType(GroovyCompile).configureEach {
options.fork = true
// A sets repro.base=A via a CommandLineArgumentProvider
options.forkOptions.jvmArgumentProviders.add({ ['-Drepro.base=A',
"-Drepro.log=${reproLog}".toString()] } as CommandLineArgumentProvider)
}
```
### `moduleB/build.gradle`
```groovy
plugins { id 'groovy' }
repositories { mavenCentral() }
dependencies {
implementation "org.apache.groovy:groovy:4.0.28"
implementation project(':transform')
}
def reproLog =
rootProject.layout.buildDirectory.file('probe.log').get().asFile.absolutePath
tasks.withType(GroovyCompile).configureEach {
options.fork = true
// B sets repro.base=B (a DIFFERENT value) via a
CommandLineArgumentProvider
options.forkOptions.jvmArgumentProviders.add({ ['-Drepro.base=B',
"-Drepro.log=${reproLog}".toString()] } as CommandLineArgumentProvider)
}
```
### One source file per module
```groovy
// moduleA/src/main/groovy/a/A.groovy
package a
class A {}
```
```groovy
// moduleB/src/main/groovy/b/B.groovy
package b
class B {}
```
### Run
```bash
./gradlew :moduleA:compileGroovy :moduleB:compileGroovy --max-workers=1
--rerun-tasks
cat build/probe.log
```
### Expected `build/probe.log`
```
pid=<X> compiling=.../A.groovy repro.base=A
pid=<Y> compiling=.../B.groovy repro.base=B
```
### Actual `build/probe.log` (bug)
```
pid=<X> compiling=.../A.groovy repro.base=A
pid=<X> compiling=.../B.groovy repro.base=A <-- same daemon PID, B sees
A's value
```
> Note: to make the daemon reuse deterministic, run the two compile tasks in
a single
> invocation. Whether B observes `A` or `B` depends only on which task
started the shared
> daemon; the point is that B runs in a daemon configured with A's JVM
arguments.
## Suggested fix / discussion
The core inconsistency is that `forkOptions.jvmArgs` participate in
compiler-daemon reuse
identity but `forkOptions.jvmArgumentProviders` do not, even though both end
up on the forked
JVM's command line. Resolving the providers and including their output in the
`DaemonForkOptions` fingerprint (used for reuse) would make daemon reuse
correct: a daemon
would only be reused for a task whose fully-resolved fork JVM arguments
match.
If that is intended behavior (providers deliberately excluded from daemon
identity), it would
help to document it prominently, because it makes `jvmArgumentProviders`
unsafe for passing any
per-task system property when compiler daemons are shared — a subtle,
data-dependent
correctness trap.
## Workarounds we are evaluating (for reference)
- Passing per-task data as static `forkOptions.jvmArgs` instead of via a
provider (partitions
daemons, but is not configuration-cache friendly with absolute paths, and
disables daemon
reuse across modules).
- Not relying on a process-global system property to identify the module
being compiled;
deriving it from the per-compilation
`CompilerConfiguration.targetDirectory` instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
