[
https://issues.apache.org/jira/browse/GROOVY-12711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18141244#comment-18141244
]
ASF GitHub Bot commented on GROOVY-12711:
-----------------------------------------
testlens-app[bot] commented on PR #3242:
URL: https://github.com/apache/groovy/pull/3242#issuecomment-6741099501
## 🚨 All tests passed but jobs failed 🚨
### Failed Jobs without Test Failures
❌ [Build and test / lts \(17,
macos-latest\)](https://github.com/apache/groovy/actions/runs/42621520016/job/129175219224?pr=3242)
🏷️ Commit: 1a509dc4a053e6291cbdff0c58124c8b2b829143
▶️ Tests: 27113 executed
🟡 Checks: 4/24 completed
### Rerun Controls
Click the checkbox to trigger a rerun:
☑️ **Rerun jobs**
---
_Learn more about TestLens at
[testlens.app/docs](https://testlens.app/docs/features/pr-comment/)._
> Joint compilation: keep javac CLASS_OUTPUT in memory when memStub is enabled
> ----------------------------------------------------------------------------
>
> Key: GROOVY-12711
> URL: https://issues.apache.org/jira/browse/GROOVY-12711
> Project: Groovy
> Issue Type: Improvement
> Components: Compiler
> Reporter: Daniel Sun
> Assignee: Daniel Sun
> Priority: Major
>
> h2. Summary
> With {{memStub}} and {{memClassOutput}} both set to {{Boolean.TRUE}}, joint
> compilation keeps the generated Java stubs and the class files on the heap.
> The class files are published on the compilation class loader, and the
> compile needs neither a stub directory nor a class-output directory.
> h2. Background
> Joint compilation can already keep generated Java stubs in memory. The joint
> option {{memStub}} ({{CompilerConfiguration.MEM_STUB}}) has been available
> since 3.0.0. {{JavaAwareCompilationUnit}} writes those stubs at
> {{Phases.CONVERSION}}, including signatures contributed by GEP-21 transforms,
> and runs javac at {{Phases.SEMANTIC_ANALYSIS}}.
> Class files were the remaining disk step. Javac wrote {{CLASS_OUTPUT}} to a
> target directory, so the core API could not keep both the stubs and the class
> files off disk.
> {{memClassOutput}} adds that step. It takes effect only together with
> {{memStub}}, and the compilation class loader publishes the class files.
> {{groovy-lsp}} does not set the option. It still uses its own Java compiler
> and an owner-only temporary directory for javac output.
> h2. Motivation
> * Tests, IDEs, and agent tools that edit Groovy and Java together need a
> joint compile that does not write stubs or class files into the project tree
> or into a temporary directory that must be locked down and removed.
> * A stub directory and a class-output directory add latency, race with file
> watchers, and can disagree with the sources the compiler just parsed.
> * In-memory stubs were already available. Keeping the class files in memory
> as well completes the joint cycle inside groovy-core. Later introspection
> tools can build on that cycle, and those tools remain separate work.
> h2. Goals
> * {{memClassOutput}} ({{CompilerConfiguration.MEM_CLASS_OUTPUT}}) is an
> explicit joint option, new in 7.0.0. It is honored only when {{memStub}} is
> also {{Boolean.TRUE}}. A missing target directory does not turn it on.
> * With both options set, {{JavaAwareCompilationUnit}} and the default javac
> compiler finish a mixed Groovy and Java compile with no {{stubDir}} and no
> filesystem target directory. The class bytes are published on the compilation
> class loader. Resolution reads that loader, in the order described under
> "What callers should expect".
> * {{groovyc}} and the Ant task leave the option unset, so disk {{stubDir}}
> and target-directory output remain the default.
> * The joint-compilation section of the groovyc guide documents {{memStub}}
> and {{memClassOutput}}.
> * Callers enable this through {{CompilerConfiguration}} and
> {{JavaAwareCompilationUnit}}. The file manager that stores the bytes is
> package-private, in {{org.codehaus.groovy.tools.javac}}.
> * Tests show a Groovy and Java cycle with no stub directory and no
> class-output directory, a javac type error on that path, and the unchanged
> disk path.
> h2. Non-Goals
> * groovy-core does not gain an MCP server, a JSON-RPC API, or an MCP library,
> and it does not change the LSP protocol.
> * {{groovy-lsp}} and {{groovy-mcp}} stay as they are. {{groovy-lsp}} still
> uses its own Java compiler and an owner-only temporary directory. Only
> {{JavacJavaCompiler}} implements {{memClassOutput}}, so using the option
> there is separate work.
> * This issue does not add an overlay filesystem or a multi-tenant sandbox.
> * Dynamic Groovy is not switched over to static type checking.
> * Builds that leave the option unset keep disk {{stubDir}} and
> target-directory output. The system property {{groovy.mem.stub}} enables
> in-memory stubs only.
> * GEP-21 stub generation, including {{MemJavaFileObject}}, stays as it is.
> Signature changes that must be visible to javac still come from the GEP-21
> {{Phases.CONVERSION}} stubbers. Other AST transforms are not moved ahead of
> stub emission.
> h2. Enabling the option
> {{memClassOutput}} ({{CompilerConfiguration.MEM_CLASS_OUTPUT}}) is new in
> 7.0.0. {{CompilerConfiguration.isMemClassOutput()}} is true only when
> {{memStub}} ({{CompilerConfiguration.MEM_STUB}}) and {{memClassOutput}} are
> both {{Boolean.TRUE}}.
> {code:java}
> CompilerConfiguration config = new CompilerConfiguration();
> Map<String, Object> options = new HashMap<>();
> options.put(CompilerConfiguration.MEM_STUB, Boolean.TRUE);
> options.put(CompilerConfiguration.MEM_CLASS_OUTPUT, Boolean.TRUE);
> config.setJointCompilationOptions(options);
> {code}
> * {{memClassOutput}} without {{memStub}} fails the compile with
> {{memClassOutput requires memStub}}. The system property {{groovy.mem.stub}}
> enables {{memStub}} only.
> * Only {{JavacJavaCompiler}} implements the option. Any other
> {{JavaCompiler}} fails the compile with {{memClassOutput requires the default
> javac joint compiler}}. Using that javac implementation with a plain
> {{CompilationUnit}} fails the compile with {{memClassOutput requires
> JavaAwareCompilationUnit}}.
> * When both options are set, {{stubDir}} is not required. The compiler does
> not add {{-d}}. If the joint options still pass {{-d}}, the file manager
> consumes it and keeps the class files in memory. {{targetDirectory}} is not
> added to the compilation class loader, because an earlier classpath entry
> would hide those classes.
> h2. How classes are published
> {{JavaAwareCompilationUnit}} captures javac {{CLASS_OUTPUT}} with
> {{InMemoryJavaFileManager}}, in package {{org.codehaus.groovy.tools.javac}},
> when javac runs. It writes the Groovy class files from the same compile into
> that same map during output. Classpath and source input still come from the
> standard file manager. The manager does no network I/O. The file manager and
> the classpath store are package-private; callers enable the behavior with the
> joint option above.
> One URL attaches the map to the compilation {{GroovyClassLoader}}:
> * The protocol is {{x-groovy-mem}} and the host is empty, so hashing or
> comparing the {{URL}} does not look up a DNS name. On the Java 17 baseline a
> security manager can still require {{NetPermission("specifyStreamHandler")}}
> to install the handler.
> * The protocol is not registered with the JVM. Only URLs created for this
> classpath can be opened. A path that escapes the map is rejected. A plus sign
> in a resource name stays a plus sign.
> * The URL is appended to the loader. {{URLClassLoader}} cannot remove a URL,
> so another {{JavaAwareCompilationUnit}} on the same loader reuses the store.
> {{groovyc}} and the Ant task build one unit per run.
> The unit opens one stage on that store. Lookup prefers the stage and
> otherwise returns the last committed bytes. A class written during this
> compile is visible to later phases. A class this compile has not written is
> still read from the previous commit, when one exists.
> The stage is committed when {{compile}} returns normally after class output
> has completed. Commit adds and replaces entries and leaves every other name
> as it was. A failed compile discards the stage, and the previously committed
> classes stay in place.
> A compile that stops at semantic analysis leaves the stage open. Javac has
> already run by then. A later {{compile}} on the same unit reuses the stage
> and does not start javac again. A different unit cannot publish while that
> stage is open: its {{compile}} throws {{IllegalStateException}} with the
> message {{another compilation is publishing to this class loader}}.
> h2. What callers should expect
> * A later successful compile replaces the stored bytes of each class it
> writes. {{loadClass}} returns the class the loader has already defined. A
> fresh loader is required to run the new bytes.
> * Lookup searches the parent loader first, then URLs in insertion order. The
> in-memory URL is last. A parent loader or an earlier classpath entry is what
> resolution and {{getResource}} see. Commit does not change that order, so the
> earlier entry still hides the bytes.
> * The compile records a warning at {{WarningMessage.LIKELY_ERRORS}}, which
> the default warning level includes. The warning text contains {{hidden by}}.
> Type checking uses the visible class. The hidden bytes are committed only
> when that compile succeeds. A failure discards the stage.
> * Class files that javac produces for the Groovy stubs stay inside javac and
> are not published. The stub is an input to javac.
> * Annotation processing is turned off with {{-proc:none}} unless the joint
> options request {{processor}}, {{processorpath}}, {{processor-module-path}},
> {{proc}}, or a flag whose name starts with {{proc:}}. Generated Java sources
> ({{SOURCE_OUTPUT}}) and non-class {{CLASS_OUTPUT}} resources stay in memory.
> Module-oriented file-manager locations are not supported.
> * {{import module}} skips loader URLs whose protocol is not {{file}}. The
> groovysh package scan ignores a URL that is neither a file nor a jar, instead
> of failing the listing. The console classpath dialog lists {{file}} URLs
> only. The in-memory URL is left out of all three.
> * The decompiler caches class stubs for {{file}}, {{jar}}, and {{jrt}} URLs
> only. The in-memory URL keeps a stable identity while its bytes can change,
> so those bytes are read again on the next lookup.
> h2. Tests
> {{MemClassOutputTest}} and {{InMemoryJavaFileManagerTest}} cover:
> * a Groovy and Java cycle with no stub directory and no class-output
> directory, including a type error reported from javac
> * disk {{stubDir}} with the option off, and {{memStub}} alone still writing
> the target directory
> * {{memClassOutput}} without {{memStub}}, a joint compiler other than javac,
> and javac invoked on a plain {{CompilationUnit}}
> * a second unit on the same loader, a unit opened while a stage is still
> open, and a failed recompile that keeps the last committed bytes
> * a compile stopped at semantic analysis and then finished on the same unit,
> classes a later compile does not write again, and stub class files left
> unpublished
> * {{import module}}, a shadowed classpath entry, and a class the loader has
> already defined
> * a plus sign in a resource name, a path that escapes the map, and generated
> Java sources kept in memory
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
