[
https://issues.apache.org/jira/browse/GROOVY-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18116759#comment-18116759
]
Paul King commented on GROOVY-12411:
------------------------------------
Notes from reading the branch, kept off the PR thread to avoid lengthening it.
Two parts: a trust question I would want closed before any release, and the
tie-ins that make the case for the library seam discussed on the PR.
h2. 1. Trust: opening a folder can execute code from that folder
Read from the code, not reproduced.
{{CompilerSettings}}'s class javadoc says Grape and {{@ASTTest}} are off "so
opening a workspace does not execute or download untrusted compile-time code".
On my reading that guarantee does not currently hold:
* When the client supplies no classpath, {{WorkspaceLayout.classpathJars}} adds
every {{lib/*.jar}} and {{libs/*.jar}} under the workspace folder to the
compile classpath automatically.
* {{GroovyCompiler.loaderFor}} builds a {{GroovyClassLoader}} over that
classpath and hands it to the {{CompilationUnit}}, where it becomes the
transform loader.
* {{ASTTransformationVisitor}} enumerates
{{META-INF/services/org.codehaus.groovy.transform.ASTTransformation}} from the
transform loader and instantiates what it finds. Local transforms named by
annotations in workspace sources load the same way.
So a repository containing {{lib/anything.jar}} with a global transform runs
that transform inside the server process the moment a {{.groovy}} file is
opened. The PR clearly thought about this class of problem -- javac is run with
{{-proc:none}} precisely to stop Java annotation processors -- but the
Groovy-side equivalent is open. It matters more here than for a typical
language server because the positioning includes being a backend for AI agents,
which routinely open repositories nobody has read.
Options, roughly in order of preference:
# Never infer a classpath from workspace contents: require the client to push
it, as the design already does for everything else.
# Run global transforms only from the server's own classpath unless the client
opts in.
# Honour a client-declared trust flag, defaulting to untrusted. VS Code's
Workspace Trust is the natural client-side hook.
Whatever is chosen, the javadoc claim should be made to match it.
h2. 2. Minimum bar before a release
# Close the path above, and align the {{CompilerSettings}} javadoc with
whatever is chosen.
# Stop disabling transforms with process-global system properties.
{{GroovyCompiler.compile}} sets {{groovy.asttest.enable}} and
{{groovy.grape.enable}} around each compile. That races anything else in the
JVM -- {{ASTTestTransformation}} reads its flag on every visit -- and for Grape
it is ineffective anyway, since {{Grape}} reads the property once into a static
at class-init (the real protection is the disabled global transform, which the
PR also sets). A per-{{CompilerConfiguration}} switch for {{@ASTTest}} would be
the clean fix and is a small, separable core change. Theoretical in a dedicated
server process; a real bug once the code is embedded in the Console, groovysh
or a notebook kernel.
# Scenario tests for the above -- "a jar in {{lib/}} is not executed" is worth
more than a lot of breadth coverage.
# Release hygiene: confirm the LSP4J licence election (it appears to be dual
EPL-2.0 / EDL-1.0, and EDL is Category A, which is simpler for an ASF release),
and whether Gson newly ships via the module's runtime classpath and so needs
LICENSE/NOTICE entries. Also worth a second attempt at the PGP key newly added
to the ignored list in {{verification-metadata.xml}}.
h2. 3. Follow-ons using tools we already have
Each of these is in-process and session-aware, and none of them wants to speak
JSON-RPC to itself -- which is the case for the language-services library seam
rather than a plug-in SPI.
|| Tool || What it gains || Depends on ||
| Groovy Console | Error underlining as you type with messages on hover,
go-to-definition, outline, completion popup | Layer-1 diagnostics and
navigation; Swing painter and popup UI |
| groovysh | Static completion through not-yet-evaluated expression chains,
signature tips from groovydoc, pre-execution diagnostics | Session context;
complements rather than replaces the existing runtime-reflective completion,
which is the better oracle where a live object exists |
| groovy-jupyter | Chained completion (a known limitation of the kernel today);
JupyterLab support nearly free, since jupyterlab-lsp feeds a concatenated
virtual document and for Groovy that concatenation is a valid script | The same
session context; plus a classpath hand-off for {{@Grab}}, since the kernel
already knows what it resolved |
| groovy-typecheckers, groovy-verify | Checker results surface as editor
diagnostics | Type-checking extensions being wired into the LSP's STC pass |
| Agents | Claude Code and OpenCode can consume a language server directly, so
a {{groovylsp}} on the PATH is the integration; SDKMAN already puts it there |
Diagnostic fidelity above all -- a false positive costs an agent more than a
missing feature |
| IDEA plugin | Conformance oracle: run a shared corpus through the plugin's
resolver and through the compiler-backed services and diff the results. Also
{{ExternalAnnotator}}, which works in every edition including Android Studio
and does not depend on the LSP API | Layer-1 library plus a test harness; no
product change on either side |
| VS Code | A first-party extension would immediately be the best Groovy option
available there | Separate deliverable; also needs a documented, scriptable way
to produce a classpath from Gradle or Maven |
The shared hard part is session context: a REPL line, a console selection and a
notebook cell are all fragments, not compilation units. Each needs a synthetic
document -- accumulated imports, declarations for known variables typed from
their runtime classes, previously defined methods and classes, then the
fragment. groovysh, the Console and the Jupyter kernel need the same thing, so
it should be built once.
h2. 4. Strategic directions
* *Type-checking extension results as editor diagnostics.* The LSP's STC pass
constructs a {{StaticTypeCheckingVisitor}} directly and never calls
{{addTypeCheckingExtensions}}, so {{NullChecker}}, {{RegexChecker}},
{{FormatStringChecker}}, {{PurityChecker}}, {{SqlInjectionChecker}} and
groovy-verify are all invisible. This is the most differentiating thing a
compiler-backed server can offer, since IDEA cannot model these without running
them. groovy-verify's counterexamples map well onto LSP -- the obligation as
the message, the counterexample as {{relatedInformation}}, and a "generate
failing test" code action. For an agent, a concrete failing input is an
unusually good repair signal. Gated on the trust decision above, since an
extension is code, and it needs a slower lane than a 150ms debounce.
* *Parser error recovery.* The {{.}} that triggers completion leaves the buffer
syntactically invalid, and Parrot builds no partial {{ModuleNode}} on a syntax
error, so there is no AST at the moment it is most needed. A last-good-AST
fallback per file or placeholder patching would unblock it now; real error
recovery in the parser is the right long-term answer and would pay off across
the Console, groovysh, our syntax diagnostics and every AST tool, well beyond
the LSP.
* *Machine-readable transform descriptions produced from the compiler.*
Transforms running at CANONICALIZATION and later -- {{@Delegate}},
{{@Immutable}}, {{@TupleConstructor}}, {{@Sortable}}, records, sealed
completion -- are invisible to anything that stops earlier, which is the root
cause of lag in the IDEA plugin, the community servers and our own tooling
alike. A larger effort, but the one that attacks the cause rather than the
symptom.
> Add a first-party Language Server Protocol implementation (groovy-lsp)
> ----------------------------------------------------------------------
>
> Key: GROOVY-12411
> URL: https://issues.apache.org/jira/browse/GROOVY-12411
> Project: Groovy
> Issue Type: Improvement
> Reporter: Daniel Sun
> Priority: Major
>
> Ship an optional first-party language server ({{groovy-lsp}}) so editors and
> AI tools can offer Groovy completion, navigation, diagnostics, and
> conservative rename against the *in-tree* compiler.
> h2. Why
> Community servers lag language changes because they compile against a Groovy
> snapshot, not this repository. {{groovy-lsp}} uses the same
> {{CompilationUnit}} as the rest of Groovy, so parser and type-system work
> appear in editors automatically.
> It is a Groovy *language core* for the Language Server Protocol, not a
> workspace or IDE. The differentiator is the same compiler, a small surface,
> and edits that refuse to guess, not more IDE features.
> h2. Intended use
> * *Editor / IDE:* start {{groovylsp}} and speak stdio JSON-RPC. The client
> supplies {{groovy.classpath}} / {{groovy.sourcePaths}}.
> * *AI tools:* the same process is the semantic backend. A later
> {{groovy-mcp}} module is expected to wrap it so a model can ask what an
> identifier is without embedding the compiler.
> * *Other Apache-2.0 Groovy LSPs:* extend the compile and protocol loop
> through the incubating {{GroovyLspExtension}} SPI instead of forking it.
> Eclipse JDT / groovy-eclipse servers remain a parallel stack for a full Java
> IDE; they cannot replace this compiler through the SPI.
> h2. Out of scope
> These are strengths of other implementations and stay out on purpose:
> * A full Java language server (Java completion, Java format, Java-only
> rename). Joint *type resolution* is in scope (see below)
> * GDSL / Jenkins pipeline descriptors (a plugin may add them; this module
> does not load GDSL)
> * Invoking Gradle or Maven (no Tooling API)
> * Debugger (DAP) and test runner
> * Identifier-text rename fallbacks (rename every {{foo}} in the file)
> * Extra name-span fields on {{AnnotatedNode}} (identifier selection is
> {{Positions.toNameRange}} in groovy-lsp)
> h2. Joint Groovy and Java sources
> When the workspace contains {{.java}} files, groovy-lsp joint-compiles them
> with Groovy through in-tree {{JavaAwareCompilationUnit}} ({{memStub}}):
> Parrot to {{CONVERSION}}, in-memory Java stubs, {{javax.tools}} javac with
> annotation processing off, then Groovy {{SEMANTIC_ANALYSIS}} (no Groovy class
> generation). Open Java buffers overlay the filesystem. Javac diagnostics and
> go-to-definition cross the language boundary via {{JavaSymbolIndex}}
> ({{com.sun.source}} {{Trees}}, captured before {{generate()}}).
> That is type resolution and navigation, not a Java language server. Java
> completion, Java formatting, and Java-only rename stay with a Java LS or
> [groovy-language-server-edt|https://github.com/tomaszrup/groovy-language-server-edt].
> h2. Comparison with other Groovy language servers
> Four implementations, four jobs. *groovy-lsp* is the in-tree language core.
> The others keep the IDE-shaped work this module refuses.
> *
> [groovy-language-server|https://github.com/GroovyLanguageServer/groovy-language-server]
> (Prominic): community server for Moonshine IDE. Apache 2.0 on a Groovy
> 4.0.26 dependency.
> * [groovyls|https://github.com/trustytrojan/groovyls] (trustytrojan): Apache
> 2.0 fork of Prominic. Jenkins GDSL, forced STC on every class, Aether
> downloads. Those features are hard-wired, not a plugin.
> *
> [groovy-language-server-edt|https://github.com/tomaszrup/groovy-language-server-edt]:
> EPL 2.0, Equinox + JDT + groovy-eclipse. Full Java LS and joint IDE
> features. Complementary stack; none of its source is copied.
> ||Aspect||groovy-lsp||Prominic||groovyls||edt||
> |License|Apache 2.0|Apache 2.0|Apache 2.0|EPL 2.0|
> |Compiler|in-tree {{CompilationUnit}}|Groovy 4.0.26 jar|Groovy 5.1.1 jar|JDT
> + groovy-eclipse|
> |Phase|{{SEMANTIC_ANALYSIS}}, no Groovy
> classgen|{{CANONICALIZATION}}|{{CANONICALIZATION}} + forced STC|JDT compile|
> |Classpath|client {{groovy.classpath}}|client {{groovy.classpath}}|client +
> Aether downloads|Red Hat Java / Gradle wrapper|
> |GDSL / Jenkins|plugin, not core|no|yes|no|
> |Joint Groovy+Java|{{JavaAwareCompilationUnit}} + {{memStub}};
> def/hover|no|no|yes (JDT IDE)|
> |Rename|declarations and bound calls only|AST references + name regex|same as
> Prominic|JDT; identifier-scan fallback|
> |Extension point|incubating {{GroovyLspExtension}}|none|none (baked in)|OSGi
> / Equinox|
> h2. Extensibility as a reusable base
> {{groovy-lsp}} is meant to be the compile + protocol loop other *Apache-2.0*
> Groovy language servers sit on, not a fork point. Plugins implement
> incubating {{GroovyLspExtension}} ({{ServiceLoader}}; tests use
> {{ExtensionHost}}). There is no second hover/completion SPI: inject members
> in {{afterCompile}} so core hover, completion, navigation, and rename see
> them.
> ||Hook||For||
> |{{configure(config, session)}}|extra classpath, extra {{groovy.*}} settings
> (for example {{groovy.dependencies}})|
> |{{initialized}} / {{shutdown}}|load or release catalogs|
> |{{afterCompile(CompilationUnit)}}|GDSL injection, a forced STC pass; must
> not generate class files|
> |{{extraDiagnostics}} / {{extraCodeActions}}|plugin diagnostics and actions
> ({{ModuleNode}} is groovy-core AST)|
> |{{commands}} / {{executeCommand}}|extra workspace commands via
> {{GroovyLspSession.applyEdit}}|
> {{GroovyLspSession}} is apply-edit, document text, folders, classpath, source
> paths, extra settings, and position encoding. It is not a compile handle. A
> throwing plugin is isolated and must not compile or publish on its own. The
> built-in {{groovy.lsp.stc}} extension is the example: annotated
> {{@TypeChecked}} / {{@CompileStatic}} only.
> * *groovyls* can sit on this SPI. GDSL inject and forced-on-all-classes STC
> belong in {{afterCompile}}; Aether downloads belong in {{configure}} from
> extra settings. Semantic tokens, conservative navigation, and joint type
> resolution are already in core. That is a plugin, not a paste of groovyls
> sources.
> * *Prominic groovy-language-server* has no unique language feature left.
> Moonshine can run {{groovylsp}} and supply {{groovy.classpath}}. A
> Moonshine-only command would be a tiny plugin.
> * *groovy-language-server-edt* cannot replace {{CompilationUnit}} with JDT
> through the SPI. Joint *type resolution* is now in groovy-lsp core; edt
> remains the complementary full Java IDE (library sources, file-move imports).
> Its identifier-scan rename fallback is not adopted.
> The SPI is incubating: design may still settle. It is a seam for Apache-2.0
> reuse, not a stability promise and not a slot for an Equinox compiler.
> h2. Rename and dynamically typed Groovy
> {{MethodCallExpression.methodTarget}} is often empty in dynamic Groovy. A
> name-and-arity match is a heuristic: good enough for hover, not for rewriting
> source.
> ||Feature||When {{methodTarget}} is set||When it is not (typical dynamic
> Groovy)||
> |Hover|Bound signature|Unique arity match on a resolved receiver, labelled
> {{_inferred_}}; otherwise {{name(...)}}|
> |Definition|That method|Every arity-matching candidate with a source
> location; never a random first match|
> |Rename / prepareRename|Allowed|Refused. A heuristic must not rewrite the
> wrong overload.|
> Declarations (class, method, field, parameter, bound local) rename by
> identity (owner + name + descriptor), not the unqualified identifier. Class
> rename also renames the file when the class name matches the file stem.
> h2. AST columns (not a breaking change)
> Compiler AST columns were already 1-based. The parser writes
> {{setColumnNumber(token.getCharPositionInLine() + 1)}}; ANTLR's
> {{getCharPositionInLine()}} is 0-based, Groovy's AST is not. Authored source
> starts at column 1, the end column is exclusive, and missing positions are
> {{-1}}.
> GROOVY-12085 changed the *unit* (Unicode code points instead of UTF-16
> units). It did not change 0-based vs 1-based.
> The patch also corrects {{ASTNode}} javadoc that had said "0-indexed" (added
> 2026-05-03, never shipped in a release). Getters, setters, and stored values
> are unchanged. That is a documentation fix, not a runtime ABI change.
> LSP positions stay 0-based in the encoding negotiated at {{initialize}}
> ({{utf-16}} by default; {{utf-8}} / {{utf-32}} optional). The server
> converts; clients never see AST columns. Existing LSP clients do not need to
> change how they send positions.
> h2. What lands
> * Optional subproject {{groovy-lsp}} ({{optionalModule()}}, not
> {{groovy-all}})
> * {{groovylsp}} / {{groovylsp.bat}} on the binary distribution (stdio
> JSON-RPC)
> * LSP 3.18 over Eclipse LSP4J 1.0; unknown 3.19 fields ignored; advertise
> only implemented capabilities
> * LSP4J is EPL 2.0 Category B *binary*; its source is not mixed into Groovy
> * Compile open Groovy buffers through {{SEMANTIC_ANALYSIS}} (no Groovy class
> generation)
> * Joint compile of workspace {{.java}} via {{JavaAwareCompilationUnit}} +
> {{memStub}} and {{javax.tools}} javac
> * Incubating {{GroovyLspExtension}} ({{ServiceLoader}}). Built-in
> {{groovy.lsp.stc}} runs STC only on {{@TypeChecked}} / {{@CompileStatic}}
> members
> * {{LocatedMessage}}: {{toDiagnostic()}} / {{write()}} no longer NPE when the
> CST context is null
> * Docs: {{groovy-lsp.adoc}}, {{subprojects/groovy-lsp/ARCHITECTURE.md}}
> h2. Language features
> File types: {{.groovy}}, {{.gvy}}, {{.gy}}, {{.gsh}}, or
> {{languageId=groovy}}. Workspace {{.java}} participates in joint compile
> (diagnostics, definition, hover), not as a Java language server.
> * *Editing:* incremental sync; completion (keywords, snippets, members,
> workspace types with auto-import, GDK) and {{completionItem/resolve}}; format
> / range format / on-type indent; organize imports; unused-import hints;
> generate accessors / {{toString}} / {{equals}}/{{hashCode}} / constructor;
> add import, package, {{@Override}}, missing class stub
> * *Navigation:* definition, type definition, implementation, declaration,
> references (by identity); call and type hierarchy; document / workspace
> symbols; folding; selection range; document highlight; import document links;
> Groovy to Java and Java to Groovy definition on jointly compiled types
> * *Presentation:* push/pull diagnostics (compiler + unused import + STC on
> annotated members + javac); semantic tokens; inlay hints; reference-count
> code lenses
> * *Workspace:* folders, watched files, configuration,
> {{window/workDoneProgress}}, {{workspace/willRenameFiles}} for Groovy sources
> * *Commands:* {{groovy.lsp.organizeImports}}, {{groovy.lsp.showReferences}},
> {{groovy.lsp.gotoSuperMethod}}
> h2. Safety and classpath
> * {{@Grab}} and {{@ASTTest}} off unless the client sets {{grapeEnabled}} /
> {{astTestEnabled}}
> * Diagnostics scoped to the owning source unit (shared {{ErrorCollector}})
> * No Gradle or Maven Tooling API. If {{classpath}} / {{sourcePaths}} are
> empty, fall back to {{src/main/groovy}}, {{src/main/java}} (and friends) and
> jars under {{lib}} / {{libs}}
> * Unchanged inputs skip recompilation; {{GroovyClassLoader}} is reused and
> closed on shutdown
> * Joint javac output is a per-compile directory under {{.groovy/lsp-javac}},
> owner-only where POSIX allows, deleted on the next compile
> * A throwing plugin must not abort compile or a request
> h2. How to run
> {code}
> groovylsp
> groovylsp --stdio
> groovylsp --socket <port>
> groovylsp --help
> groovylsp --version
> {code}
> Do not write to stdout from Groovy code running inside the server; the
> launcher redirects {{System.out}} to {{System.err}} so stray prints cannot
> break the protocol stream.
> h2. See also
> * GROOVY-12085 (code-point columns; not an origin change)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
