This is an automated email from the ASF dual-hosted git repository.
paulk-asert pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/groovy-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new c56c304 updated with recent changes for GROOVY-12151
c56c304 is described below
commit c56c304f84c848606c9e7642549653841dea065a
Author: Paul King <[email protected]>
AuthorDate: Wed Jul 15 23:42:01 2026 +1000
updated with recent changes for GROOVY-12151
---
site/src/site/wiki/GEP-27.adoc | 99 +++++++++++++++++++++++++++++-------------
1 file changed, 70 insertions(+), 29 deletions(-)
diff --git a/site/src/site/wiki/GEP-27.adoc b/site/src/site/wiki/GEP-27.adoc
index 87d4d98..e7dff31 100644
--- a/site/src/site/wiki/GEP-27.adoc
+++ b/site/src/site/wiki/GEP-27.adoc
@@ -14,7 +14,7 @@
*Comment*:: Reduces the per-closure/per-lambda generated-class explosion by
hoisting bodies onto the enclosing class. The sound-by-construction SAM-lambda
slice is merged for Groovy 6; closure packing (S1) is implemented and proposed
as a Groovy 6 stretch goal (GROOVY-12151); object elision and writer
unification remain for Groovy 7. Strictly additive; gated by a flag for
back-out.
*Leader*:: Paul King
*Created*:: 2026-07-09
-*Last modification*:: 2026-07-15
+*Last modification*:: 2026-07-16
****
[width="80%",align="center"]
@@ -47,8 +47,8 @@ actually needs:
* a *SAM-targeted lambda* becomes a `LambdaMetafactory` functional-interface
instance over the
hoisted method — no generated class at all, and a zero-allocation singleton
when
non-capturing (Java parity);
-* a *closure* becomes one shared `Closure` adapter that dispatches to the
hoisted method — no
- per-closure class, while remaining a real `groovy.lang.Closure` (so `curry`,
`memoize`,
+* a *closure* becomes a shared `Closure` adapter (a small fixed-arity family)
that dispatches to
+ the hoisted method — no per-closure class, while remaining a real
`groovy.lang.Closure` (so `curry`, `memoize`,
`trampoline`, and iteration continue to work);
* where a value neither escapes nor needs closure identity, the object can be
elided entirely.
@@ -193,11 +193,15 @@ A hoisted representation loses exactly three things
relative to a normal generat
time against the owner; a normal closure resolves free names against a
`delegate` set at
runtime. This is the one true soundness boundary (see _The soundness
boundary_).
* *Serialization.* A dispatch-table-backed adapter is not portably
serializable the way a
- generated closure class is.
-* *Per-literal class identity.* All packed closures share one adapter class, so
- `closure.getClass()` no longer distinguishes literals, and class-level
metaclass changes on
- the adapter are global to packed closures (per-instance `setMetaClass` is
fully honoured —
- see _MOP transparency_).
+ generated closure class is. (`dehydrate()`/`rehydrate()` themselves keep
working — dispatch
+ holds its own receiver, so a dehydrated packed closure stays callable — but
the adapter cannot
+ be written to an object stream.)
+* *Per-literal class identity.* Packed closures share a small *fixed-arity
adapter family*
+ (`PackedClosure$Fixed0..4`, `$FixedIt` for implicit-parameter literals,
`$FixedN` for higher
+ and vararg arities), so `closure.getClass()` distinguishes arity — enough
for class-level
+ introspection such as SAM-overload selection — but not individual literals,
and class-level
+ metaclass changes on a family member affect every packed closure of that
arity (per-instance
+ `setMetaClass` is fully honoured — see _MOP transparency_).
So "needs closure identity" is not a broad lattice; it is essentially the
predicate *"does
externally-set delegate/resolveStrategy or serialization reach this value?"*
@@ -247,7 +251,7 @@ Given the required capability, the compiler picks the
lightest representation th
|Strategy |When |Result
|*S0* full generated class |needs external delegate / serialization, or
capability unprovable |today's behaviour — one class per closure/lambda
-|*S1* shared `Closure` adapter |closure-shaped, no external
delegate/serialization |hoisted body + one shared adapter; no per-closure
class; `curry`/`memoize` preserved
+|*S1* shared `Closure` adapter |closure-shaped, no external
delegate/serialization |hoisted body + a fixed-arity adapter family; no
per-closure class; `curry`/`memoize` preserved
|*S2* metafactory SAM |functional-interface target, no closure identity needed
|hoisted body + `LambdaMetafactory`; **no class**; zero-alloc singleton when
non-capturing; static-only
|*S3* elide / inline |value neither escapes nor needs an object (e.g.
call-once) |no object at all
|===
@@ -363,14 +367,18 @@ modes:
A conservative *escape gate* shrinks the runtime surface further and is
implemented in the S1 slice: a
closure literal that visibly escapes its method — stored into a
field/property/index (the Grails
-`static constraints = { … }` shape), returned, appended with `<<`, or placed
in a collection literal
-— is declined at compile time and kept as a class. This moves the clearest
delegate-risk cases from
+`static constraints = { … }` shape), *initialising a field* (`def action = { …
}` at class level,
+which compiles outside the enclosing method's visible code and is checked
against the class's
+fields directly), returned, appended with `<<`, or placed in a collection
literal — is declined at
+compile time and kept as a class. This moves the clearest delegate-risk cases
from
a runtime throw to a compile-time non-event; those declines are also what
`WARN`/`STRICT` report.
A sibling gate applies the same discipline to the serialization boundary: a
literal that is visibly
-serialization-bound — cast or coerced to a `Serializable` type, or passed
directly to a
-`writeObject` call — declines and keeps its class (so it serializes as
before), with transitive
-routes left to the adapter's fail-fast `writeObject`, exactly as transitive
escapes are left to the
-delegate guard.
+serialization-bound — cast or coerced to a `Serializable` type, passed
directly to a
+`writeObject` call, or held in a *local* that is (the canonical
+`def c = { … }; out.writeObject(c)` shape, tracked one def-use hop within the
method) — declines
+and keeps its class (so it serializes as before), with genuinely transitive
routes (helper methods,
+constructor-mediated graphs) left to the adapter's fail-fast `writeObject`,
exactly as transitive
+escapes are left to the delegate guard.
Because syntactic escape analysis cannot prove that, say, a custom `each` does
not set a delegate,
the runtime guard remains the essential backstop under dynamic compilation.
@@ -458,9 +466,11 @@ behind the same flag; only mutated-capture lambdas remain
on the class-based pat
=== Stretch goal (implemented): closure packing (S1)
-A shared `Closure` adapter (`PackedClosure`) that dispatches to a hoisted
body, applied to
-closures rather than lambdas — now implemented in the code generator
(GROOVY-12151) and proposed
-for Groovy 6.0 as a stretch goal. It covers the hard cases: no-capture and
captured closures (by
+A shared `Closure` adapter family (`PackedClosure`, instantiated as the
fixed-arity member
+matching the literal — `Fixed0..Fixed4`, `FixedIt` for implicit-parameter
literals, `FixedN` for
+higher and vararg arities) that dispatches to a hoisted body, applied to
closures rather than
+lambdas — now implemented in the code generator (GROOVY-12151) and proposed
for Groovy 6.0 as a
+stretch goal. It covers the hard cases: no-capture and captured closures (by
value; written captures via shared `Reference`, including `+=`/`++`),
*arbitrary-depth nested
closures*, implicit `it`, under both `@CompileStatic` and dynamic compilation,
correctly declining
closures that use `delegate`/`owner`/`super`, or that carry default parameter
values (multi-arity —
@@ -492,12 +502,22 @@ diagnostics. The two headline capabilities are done:
instance's metaclass is stock and no category is active (one latched-boolean
check, the same
conditions the classic call-site caches test), and anything MOP-relevant
routes through
`invokeMethod`. Interception via a per-instance metaclass is honoured
identically on the dynamic
- and Java/GDK paths, matching generated closure classes. The adapter has its
own registered stock
- metaclass, `PackedClosureMetaClass` — the analog of `ClosureMetaClass` for
generated closures —
- giving reflection-free `call`/`doCall` dispatch on the MOP route,
`respondsTo` answers faithful to
- the instance's declared parameter types, and the same category stance as
`ClosureMetaClass`. (The
- `GeneratedClosure` marker is deliberately not used: its metaclass machinery
assumes an inner class
- of the owner, which a shared top-level adapter is not.)
+ and Java/GDK paths, matching generated closure classes. The adapter family
has its own registered
+ stock metaclass, `PackedClosureMetaClass` — the analog of `ClosureMetaClass`
for generated
+ closures — giving reflection-free `call`/`doCall` dispatch on the MOP route,
`respondsTo` answers
+ faithful to the instance's declared parameter types, and the same category
stance as
+ `ClosureMetaClass`. Each fixed-arity member carries the `GeneratedClosure`
marker and declares
+ exactly the `doCall` signature(s) its generated-class counterpart would
(including the fuzzy
+ "0 or 1" pair for implicit-parameter literals), so *class-level*
introspection — SAM-overload
+ selection by closure arity, `ClosureMetaMethod` construction,
`MetaClassImpl` method selection
+ under a replaced metaclass — behaves exactly as with generated classes; the
general dispatch
+ entry is deliberately not named `doCall`, so no varargs signature exists for
MOP selection to
+ mistake a single `Object[]` argument for an argument list. Argument
adaptation replicates
+ metaclass dispatch bit for bit (pinned empirically against classed
closures): arity mismatches
+ raise `MissingMethodException`, a single `List` destructures across a
non-one-parameter
+ signature, trailing-array parameters vararg-collect, and per-parameter
coercion follows the
+ metaclass compatibility matrix (`BigDecimal`→`double` yes,
`String`→`Integer` no,
+ `Closure`→SAM yes).
The API surface (`@PackedClosures`, `PackMode`) is marked `@Incubating`. What
keeps this a stretch
goal rather than a committed 6.0 deliverable is that it is opt-in only (the
automatic path is not
@@ -553,10 +573,31 @@ a goal; this is tracked as a separate initiative.
Flip closure packing from experimental (opt-in `groovy.target.closure.pack`)
to default-on. This is
a policy decision gated on the remaining human-only tooling verification (see
_IDE and tooling
-considerations_) and on migrating the tests that assert the old bytecode shape
— not on further
-mechanism work: the dispatch and MOP-transparency mechanisms are complete and
verified (interception,
-categories, and introspection behave identically to generated closure classes
on every path), and the
-remaining documented residuals are those listed under _The one that matters:
required capability_.
+considerations_) — not on further mechanism work: the dispatch and
MOP-transparency mechanisms are
+complete and verified (interception, categories, and introspection behave
identically to generated
+closure classes on every path), and the remaining documented residuals are
those listed under
+_The one that matters: required capability_.
+
+==== Hardening under maximal packing (done)
+
+The strongest evidence for the flip comes from running *Groovy's entire own
test suite with the
+flag forced globally on* — core, the test corpus, and everything the tests
compile and evaluate at
+runtime — deliberately packing far beyond any real configuration to find where
theory and MOP
+reality diverge. The campaign started at **39 failing test classes (~140
failures)** and ended at
+**one** (a single context-dependent case that passes in every standalone
reproduction). Everything
+in between was resolved by root-cause fixes that *widened* packed coverage
rather than narrowing
+it: the fixed-arity adapter family (restoring class-level arity introspection
for SAM-overload
+selection), argument-adaptation parity pinned empirically against classed
closures, holder
+threading for captures re-captured by nested closures, flow-inferred capture
typing, two escape-
+and serialization-gate extensions (field initialisers; one-hop locals into
`writeObject`), the
+mixed-mode dynamic-island decline, and closure-context contracts re-homed with
the body (dynamic
+property fallback for cross-class private access, `Map`-owner entry semantics,
inferred return
+coercion). Tests that assert the *pre-pack generated-class bytecode shape* are
conditionally
+skipped under the flag with named reasons (they run unchanged by default), and
tests whose
+*subject* is closure-class serialization declare `@PackedClosures(mode =
DISABLED)` fixtures —
+every behavioural test in the suite runs and passes packed. The methodology
(forced-global flag,
+per-class JVM isolation, compile-time bytecode verification harnesses, and
pinning classed
+behaviour empirically before replicating it) is repeatable for future slices.
== Configuration and flags
@@ -834,7 +875,7 @@ stretch goal) builds on the shared backend proven in phases
1–2; phases 4–5
|Changing closure/lambda _semantics_ |Not planned |This is a codegen/packaging
optimisation only. Eligible closures compute identical results and keep their
reachable `Closure` API; ineligible ones are compiled exactly as today.
|Packing closures that use externally-set `delegate`/`resolveStrategy` |Not
planned |These require a real per-closure representation; they are detected
(statically) or declined and fall back to S0.
-|Serializable closure/lambda hoisting |Deferred |Serializable lambdas and
_visibly_ serialization-bound closures (cast/coerced to a `Serializable` type,
or passed directly to `writeObject`) are kept on the class-based path; a packed
closure reached by a transitive serialization route fails fast at runtime with
an actionable message. Hoisting the serializable cases would require carrying
the deserialisation machinery to the enclosing class.
+|Serializable closure/lambda hoisting |Deferred |Serializable lambdas and
_visibly_ serialization-bound closures (cast/coerced to a `Serializable` type,
passed directly to `writeObject`, held in a local that is, or initialising a
field) are kept on the class-based path; a packed closure reached by a
genuinely transitive serialization route fails fast at runtime with an
actionable message. Hoisting the serializable cases would require carrying the
deserialisation machinery to the enclosin [...]
|Packing default-parameter (multi-arity) closures |Deferred |A single hoisted
method cannot reproduce the arity-overloaded dispatch a default parameter
generates, so these decline to S0 today. A future slice would hoist one method
per arity (reusing the existing default-fill codegen) and claim one
dispatch-table id per arity — the per-arity tables the shipped adapter already
uses, and the same primitive the _MethodClosure_ alignment above would share.
|Object elision (S3) |Deferred (Groovy 7) |The most aggressive step; needs
escape proof and, for iteration, callee knowledge.
|Dynamic-mode automatic packing of _free-name_ closures |Not planned |A
closure that resolves a free name cannot be proven delegate-safe without types;
that subset stays opt-in via `@PackedClosures` (+ runtime guard) under dynamic
compilation.
