daniellansun commented on PR #2736: URL: https://github.com/apache/groovy/pull/2736#issuecomment-5145783602
# GROOVY-12191 Performance Verification Report V4 **Scoped indy SwitchPoint invalidation — exact-class stock policy (no hierarchy fan-out)** | Item | Value | |---|---| | Issue | [GROOVY-12191](https://issues.apache.org/jira/browse/GROOVY-12191) | | Commit under test | `c81e62bccaad66e6d4816884ea40807d3a5ab199` | | Baseline (parent) | `0ca665dad0d4070bfa218d9b5577ec423c8b2181` | | Date | 2026-08-01 | | Verdict | **PASS** — cross-type MetaClass churn no longer globally deoptimizes hot monomorphic call sites; parent MetaClass churn no longer force-relinks subtype sites under stock policy; steady-state hot path is parity; correctness suite green (**94/94**) | --- ## 1. Executive summary This verification compares the **final GROOVY-12191 stack** at `c81e62bcca` against the pre-scoping parent `0ca665dad0` (process-wide MOP `SwitchPoint`). At HEAD, indy MOP invalidation uses: - **One SwitchPoint domain per MetaClass instance** (weak identity map in `IndyInvalidation`). - **ClassInfo pending domain** only for pre-MetaClass link (defineClass safety); first MetaClass install retires it. - **Two-width policy only:** - **Exact class** — stock `MetaClassImpl` / EMC changes (`invalidateClass`), including `incVersion` and per-instance MetaClass. - **All loaded** — category enter/leave, unscoped registry events, and non-stock custom MetaClass kinds. - **No hierarchy SwitchPoint fan-out** and **no `ClassHierarchyIndex`** — ancestor EMC visibility on linked miss sites is live via `MetaClass.invokeMethod` / property-miss hierarchy walks (PR #2736 / paulk-asert + blackdrag review). - Linked call sites still carry **one** `SwitchPoint.guardWithTest` — the pre-6.0 monomorphic hot-path shape. On the same host, JDK, and JMH annotation settings, A/B measurement of **HEAD (`c81e62bcca`) vs baseline (`0ca665dad0`)** shows: 1. **Near-zero regression on steady-state monomorphic hot paths** (baseline ratios ≈ **1.01–1.07×**, within noise). 2. **Large gains under cross-type MetaClass churn** — the primary win: - `CallSiteInvalidationBench.crossTypeInvalidationEvery1000`: **428.0 → 2.92 ms/op** (≈ **147×**). - `ScopedInvalidationBench.hotLoop_unrelatedMetaClassChurn`: throughput ≈ **211×**. 3. **Framework-style “unrelated burst then steady request loop”** (`burstThenSteadyState`): ≈ **101×**. 4. **Parent MetaClass churn no longer deopts subtype hot sites** (stock exact-class): - `parentChild_parentMetaClassChurn`: ≈ **130×** vs parent (parent collapsed to global-deopt floor; HEAD pays mainly MetaClass-write cost, similar to unrelated churn). 5. **Same-type invalidation and Category semantics are not “falsely optimized”**: same-type churn remains far slower than baseline (forced re-link); category enter/leave stays bulk and matches parent throughput (~1×). 6. **Correctness: 94 passed / 0 failed** (scoped-invalidation suite on HEAD; hierarchy-index tests removed with the index). > In one line: without adding a second hot-path guard, the change shrinks the blast radius of unrelated (and parent) MetaClass mutations from process-global to **exact class** for stock metaclasses, with measured gains on the order of **10¹–10²×**, while preserving bulk semantics for Category and custom MetaClass kinds. --- ## 2. Change mechanism and performance hypotheses ### 2.1 Previous model (baseline `0ca665dad0`) - `IndyInterface.switchPoint` was a **process-wide shared** `SwitchPoint`. - Any MetaClass registry change → `invalidateSwitchPoints()` → **every** linked indy site’s guard failed → mass re-link. - Typical symptom: a framework/plugin extends MetaClass on *ColdType* at startup or hot-reload; application hot paths on *HotTarget.compute()* were deoptimized together with it. The same collapse applied when a **parent** type’s MetaClass changed — all process sites paid re-link tax. ```text // baseline (simplified) protected static SwitchPoint switchPoint = new SwitchPoint(); // MetaClassRegistry listener → always: synchronized (IndyInterface.class) { SwitchPoint old = switchPoint; switchPoint = new SwitchPoint(); SwitchPoint.invalidateAll(new SwitchPoint[]{old}); } ``` ### 2.2 New model (`c81e62bcca`) | Component | Role | |---|---| | `SwitchPointInvalidator` | Single-domain lifecycle: CAS `get` / `detachLive` / `invalidate` | | `IndyInvalidation` | Policy + weak identity map `MetaClass → SwitchPointInvalidator`; exact-class vs all-loaded bulk | | `ClassInfo` | Pending domain for pre-MetaClass link only; version bumps → exact-class invalidation | | `Selector` / `IndyCompoundAssign` / `ColdReflectiveMethodHandleWrapper` | Install MetaClass-domain (or pending) SwitchPoint guard at link time | **Hot-path shape (intentionally monomorphic):** ```text handle = mopSwitchPoint.guardWithTest(fast, fallback) ``` Still a single guard—matching the pre-6.0 monomorphic shape. There is **no** second category SwitchPoint on the hot path. **Invalidation policy (two widths):** | Event | Behavior | Expected hot-path cost | |---|---|---| | Unrelated-type MetaClass change (stock) | Retire **that class’s** domain only | Hot sites **do not** re-link | | Same-type MetaClass change (stock) | Retire that class’s domain | Must re-link (correctness) | | Parent EMC / stock MetaClass change | Exact class of **parent only** — **no** subtype SwitchPoint fan-out | Subtype sites stay linked; live miss route observes ancestor EMC | | Non-stock custom MetaClass registry event | Process-wide bulk (`invalidateBulk`) | Correctness-first, rare | | Category enter/leave, `invalidateCallSites()` | Bulk-retire all loaded class domains | Same order of cost as old global invalidation | | Unattributed MetaClass event | `invalidateUnscoped()` bulk | Conservative, correctness-first | **What was deliberately removed after PR #2736 review (vs intermediate stack at `637bdaf`):** - `ClassHierarchyIndex` and parent→child SwitchPoint fan-out for stock `MetaClassImpl`/EMC. - Dual exact APIs / multi-width matrix residue — collapsed to exact vs bulk. ### 2.3 Hypotheses under test | ID | Hypothesis | How verified | |---|---|---| | H1 | Baseline hot loop has no material regression | A/B ratio ≈ 1 | | H2 | Cross-type churn throughput/latency improves substantially | Unrelated ≫ parent; on HEAD, unrelated ≫ same-type | | H3 | Pure hot loop after unrelated burst approaches baseline | `afterUnrelatedBurst` ≈ `baseline` | | H4a | Same-type / category still pay re-link tax | Clearly slower than baseline | | H4b | Parent MetaClass churn does **not** force-relink subtype sites (stock exact-class) | `parentChild_parentMetaClassChurn` ≫ parent floor; ratio to `parentChild_baseline` similar to unrelated-vs-baseline | | H5 | Correctness suite stays green | Unit / integration tests | --- ## 3. Methodology ### 3.1 Environment | Item | Value | |---|---| | OS | Linux 6.15.5 x86_64 (hostname: `hera`) | | CPU | 6 vCPUs visible | | Memory | 23 GiB | | JDK | Amazon Corretto **25.0.2** (`25.0.2+10-LTS`) | | Build | Gradle wrapper, `./gradlew :perf:jmh`, indy enabled by default | | Trees | HEAD worktree `@ c81e62bcca`; baseline worktree `@ 0ca665dad0` | | Bench source parity | `ScopedInvalidationBench` copied into parent worktree for fair A/B (not present on parent history); `CallSiteInvalidationBench` workloads identical | | Run order | Sequential: HEAD scoped → HEAD callsite → parent scoped → parent callsite (avoid dual-JVM core contention); no CPU pinning | ### 3.2 Benchmark suites 1. **`org.apache.groovy.bench.ScopedInvalidationBench`** (Throughput, ops/ms) - 100 000 monomorphic calls per op; MetaClass write every 1000 iterations in churn cases. - JMH: `@Warmup(3×2s) @Measurement(5×2s) @Fork(2)` → 10 samples per benchmark. - Includes `parentChild_*` rows (stock exact-class: parent churn must **not** deopt child). 2. **`org.apache.groovy.perf.grails.CallSiteInvalidationBench`** (AverageTime, ms/op) - Same 100 000-iteration loops; cross-type / same-type / burst patterns. - Same JMH annotation settings. ### 3.3 Correctness ```bash ./gradlew :test --tests Groovy12191 \ --tests org.apache.groovy.runtime.indy.IndyInvalidationTest \ --tests org.apache.groovy.runtime.indy.SwitchPointInvalidatorTest \ --tests org.codehaus.groovy.vmplugin.v8.IndyScopedSwitchPointTest --rerun-tasks ``` **Result (HEAD `c81e62bcca`): 94 passed / 0 failed / 0 skipped / 0 error.** | Test class | Cases | |---|---:| | `bugs.Groovy12191` | 29 | | `IndyInvalidationTest` | 41 | | `SwitchPointInvalidatorTest` | 12 | | `IndyScopedSwitchPointTest` | 12 | | **Total** | **94** | > Note: intermediate PR heads included `ClassHierarchyIndexTest` (~12 cases). That index and its tests are **intentionally gone** at `c81e62bcca`. Behavioural coverage of live miss / property / array lattice lives in `Groovy12191` and unit policy tests. ### 3.4 Statistics and artifacts - Scores are JMH means; `±` is JMH’s default **99.9% CI**. - **Throughput** ratio = HEAD / parent (>1 is better). - **AverageTime** speedup = parent / HEAD (>1 is better). - Conclusions rest on **order-of-magnitude and structural separation**, not ±1% micro-deltas. Raw JSON (local run artifacts, not committed): - `/tmp/groovy-12191-perf-c81e62bcc/head/scoped-results.json` - `/tmp/groovy-12191-perf-c81e62bcc/parent/scoped-results.json` - `/tmp/groovy-12191-perf-c81e62bcc/head/callsite-results.json` - `/tmp/groovy-12191-perf-c81e62bcc/parent/callsite-results.json` - `/tmp/groovy-12191-perf-c81e62bcc/summary.json` --- ## 4. Results ### 4.1 ScopedInvalidationBench (Throughput, ops/ms — higher is better) | Benchmark | Parent | HEAD | HEAD/Parent | Interpretation | |---|---:|---:|---:|---| | `hotLoop_baseline` | 3.847 ± 0.207 | 4.017 ± 0.364 | **1.04×** | H1: no hot-path regression | | `hotLoop_afterUnrelatedBurst` | 3.798 ± 0.677 | 4.432 ± 0.229 | **1.17×** | H3: full speed after unrelated burst | | `hotLoop_unrelatedMetaClassChurn` | 0.00243 ± 0.00018 | 0.512 ± 0.147 | **≈211×** | **H2 primary win** | | `hotLoop_sameTypeMetaClassChurn` | 0.00252 ± 0.00011 | 0.0188 ± 0.0012 | **≈7.4×** | Still forced re-link; narrower blast radius than parent | | `hotLoop_categoryEnterLeave` | 0.00238 ± 0.00022 | 0.00230 ± 0.00034 | **≈0.97×** | H4a: bulk semantics retained (parity within noise) | | `parentChild_baseline` | 3.234 ± 0.383 | 2.889 ± 0.068 | 0.89× | Both full-speed order; container noise | | `parentChild_parentMetaClassChurn` | 0.00242 ± 0.00027 | 0.316 ± 0.034 | **≈130×** | **H4b: parent churn no longer global-deopts child** | **Structural fingerprint on HEAD only:** ```text baseline ≈ 4.02 ops/ms after burst ≈ 4.43 ops/ms (≥ baseline: no residual deopt) unrelated churn ≈ 0.51 ops/ms (~8× slower than baseline: mostly MetaClass write cost) same-type churn ≈ 0.019 ops/ms (~214× slower than baseline: write + same-class re-link) category ≈ 0.002 ops/ms (bulk re-link tax) parentChild baseline ≈ 2.89 ops/ms parentChild + parent ≈ 0.32 ops/ms (~9× slower than parentChild baseline — write-dominated, not re-link floor) ``` On parent, `unrelated`, `same-type`, and `parentChild_parentMetaClassChurn` **all collapse to ~0.002**, showing that the old global SwitchPoint made every MetaClass mutation equivalent to “global deopt.” HEAD separates: - **unrelated vs same-type** by ~**27×** (0.512 / 0.019) — scoping is live. - **parentChild parent-churn vs parent global floor** by ~**130×** — stock exact-class means parent EMC writes no longer retire the child’s SwitchPoint. ### 4.2 CallSiteInvalidationBench (AverageTime, ms/op — lower is better) | Benchmark | Parent | HEAD | Speedup (P/H) | Interpretation | |---|---:|---:|---:|---| | `baselineHotLoop` | 0.260 ± 0.022 | 0.254 ± 0.014 | **1.02×** | H1 | | `baselineListSize` | 0.241 ± 0.008 | 0.240 ± 0.007 | **1.01×** | H1 | | `baselineSteadyStateNoBurst` | 0.608 ± 0.022 | 0.589 ± 0.040 | **1.03×** | H1 | | `baselineMultipleCallSites` | 20.86 ± 1.48 | 19.41 ± 1.46 | **1.07×** | H1 | | `crossTypeInvalidationEvery10000` | 301.3 ± 20.4 | 2.462 ± 0.360 | **≈122×** | H2 | | `crossTypeInvalidationEvery1000` | 428.0 ± 35.2 | 2.921 ± 0.596 | **≈147×** | **H2 primary win** | | `crossTypeInvalidationEvery100` | 134.8 ± 6.6 | 7.393 ± 0.250 | **≈18×** | Still large under denser churn | | `listSizeWithCrossTypeInvalidation` | 311.6 ± 26.5 | 1.657 ± 0.466 | **≈188×** | JDK-type hot sites benefit too | | `multipleCallSitesWithInvalidation` | 1081 ± 89 | 24.93 ± 3.42 | **≈43×** | Multi-site still much improved | | `burstThenSteadyState` | 141.9 ± 19.3 | 1.406 ± 0.203 | **≈101×** | “Framework MC extend + request steady state” | | `sameTypeInvalidationEvery1000` | 401.4 ± 31.0 | 51.58 ± 4.15 | **≈7.8×** | Same-type still slow; local invalidation cheaper than global rotate | **Penalty vs baseline on HEAD:** | Scenario | Relative cost | Meaning | |---|---:|---| | crossType @1000 | 2.92 / 0.25 ≈ **12×** `baselineHotLoop` | Dominated by ~100 ColdType MetaClass writes, not hot-site deopt | | sameType @1000 | 51.6 / 0.25 ≈ **203×** `baselineHotLoop` | Clear re-link tax | | burstThenSteady | 1.41 / 0.59 ≈ **2.4×** steady baseline | Burst write cost dominates; steady calls recovered | On parent, crossType@1000 and sameType@1000 are **both ~400+ ms/op**, again confirming global invalidation collapsed cross-type into full deopt. --- ## 5. Why it is faster ```text Old path New path (stock) ColdType.metaClass.foo = ... ColdType.metaClass.foo = ... │ │ ▼ ▼ invalidate global SwitchPoint retire MetaClass(ColdType) domain only │ │ ▼ ▼ ALL sites: guard fails ColdType sites only HotTarget.compute → full re-select HotTarget.compute → stays linked (MH rebuild / re-guard / cold path) (single SwitchPoint check passes) Parent.metaClass.bar = ... Parent.metaClass.bar = ... │ │ ▼ ▼ global SwitchPoint (same collapse) retire Parent domain only │ Child linked sites stay warm ▼ Live miss path sees Parent EMC Child sites re-link (unnecessary) without SwitchPoint fan-out ``` Cost breakdown: 1. **Avoided re-links** — After each global invalidation, hot sites re-run `Selector`, reinstall guards, and rewrite `CallSite`s. At 100 churn events × many sites this dominates (parent’s 400+ ms/op). 2. **Avoided JIT churn** — Frequent process-wide `SwitchPoint.invalidateAll` breaks monomorphic inlining assumptions and amplifies deopt/recompile cost. 3. **Unchanged hot-path guard count** — Still one `guardWithTest`, so baselines stay flat (H1). 4. **Same-type still correctly charged** — Prevents “fast but wrong” semantics when the receiver’s own MetaClass changes (H4a + 94 tests). 5. **Parent EMC no longer over-invalidates subtypes** — stock exact-class policy; miss-path self-heals via live hierarchy walk (H4b; PR #2736 experiment). 6. **Category remains bulk** — No second hot-path guard; category visibility stays simple and correct; cost matches the old global path (measured ~1.0×). Same-type still improves ~**8×** on HEAD vs parent, not because re-link is skipped, but because the invalidation surface shrinks from “all process sites” to “this class” and batch `invalidateAll` is cheaper—a secondary benefit, not the primary goal. --- ## 6. Risks and boundaries | Topic | Assessment | |---|---| | Hot-path regression | **Not observed** (baselines 1.01–1.07×) | | Semantic regression | 94/94 scoped suite green; Category / per-instance / MetaClass identity domain / live miss & property probes covered | | Dense Category enter/leave | **Still expensive** (by design); Category-heavy workloads do not get faster | | Parent EMC / hierarchy | **No SwitchPoint fan-out** for stock; visibility via live miss walk. Construction-time MetaClassImpl snapshots remain pre-existing MOP behaviour | | Custom non-`MetaClassImpl` | Process-wide bulk on registry replace (correctness-first, rare) | | Legacy `IndyInterface.switchPoint` | **Removed** on this branch (`vmplugin` internal-by-intent); external code must use `IndyInvalidation` | | Environment noise | Shared/container CPUs; conclusions use order-of-magnitude gaps (10¹–10²×), robust to ±10% noise | | Not covered here | Concurrent multi-thread churn throughput, full Grails end-to-end, JDK 17/21 matrix (recommended for CI) | --- ## 7. Hypothesis scorecard | Hypothesis | Result | Evidence | |---|---|---| | H1 baseline parity | **Holds** | Scoped baseline 1.04×; CallSite baselines 1.01–1.07× | | H2 cross-type speedup | **Holds** | ≈211× thrpt; ≈147× avgt (@1000) | | H3 post-unrelated-burst full speed | **Holds** | `afterUnrelatedBurst` ≥ baseline on HEAD | | H4a necessary invalidation still paid | **Holds** | same-type / category ≪ baseline | | H4b parent churn does not fan out | **Holds** | parentChild parent-churn ≈130× vs parent floor; write-dominated vs child baseline | | H5 correctness | **Holds** | 94/94 tests | --- ## 8. Comparison to V3 report (`637bdaf` stack) | Aspect | V3 (`637bdaf`) | V4 (`c81e62bcca`, this report) | |---|---|---| | Per-MetaClass SP domains | Yes | Yes (unchanged win) | | Hierarchy fan-out / `ClassHierarchyIndex` | Yes (EMC / array / interface) | **Removed** for stock | | Cross-type win | ~151–171× | **~147–211×** (same order) | | Parent→child SP on parent EMC | Forced child re-link | **Child SP stays live**; live miss path | | API surface | Multi-width matrix | **Exact vs bulk only** | | Correctness count | 107 (incl. index tests) | **94** (index tests gone; more behaviour probes) | The primary performance story (**cross-type deopt removal**) is unchanged. V4 additionally removes inert hierarchy invalidations, which **improves** parent-churn isolation for stock metaclasses and simplifies the machine. --- ## 9. Conclusions and recommendations ### Verdict `c81e62bcca` ([GROOVY-12191](https://issues.apache.org/jira/browse/GROOVY-12191)) keeps a **single-guard monomorphic hot path** while scoping indy MOP invalidation to **per-MetaClass domains** with **exact-class stock policy** (bulk only for Category / unscoped / custom MetaClass). For **unrelated-type MetaClass churn with hot monomorphic call sites**—a Grails/framework-shaped load—measured improvement is on the order of **10¹–10²×**. **Parent MetaClass churn** no longer force-relinks subtype sites under stock policy. Steady-state undisturbed paths show **no material regression**. Correctness costs for Category and same-type changes **remain visible and intentional**. **Performance verification: PASS.** ### Recommendations 1. Keep `ScopedInvalidationBench` (including `parentChild_*`) and the cross-type / burst rows of `CallSiteInvalidationBench` in regular JMH regression so a return to a global SwitchPoint (or accidental reintroduction of hierarchy fan-out) cannot land silently. 2. In release notes, state clearly that Category and unscoped MetaClass events may still bulk-invalidate; the win is **type-/MetaClass-local stock MetaClass change**, including parent EMC without subtype SP fan-out. 3. Optionally re-run CallSite `crossType@1000` and baselines on JDK 17/21 to confirm cross-LTS consistency. 4. Document that MOP guards are resolved via `IndyInvalidation` (MetaClass identity map + ClassInfo pending), not a process-wide `IndyInterface.switchPoint`. --- ## Appendix A — Reproduction commands ```bash # Correctness (HEAD @ c81e62bcca) ./gradlew :test --tests Groovy12191 \ --tests org.apache.groovy.runtime.indy.IndyInvalidationTest \ --tests org.apache.groovy.runtime.indy.SwitchPointInvalidatorTest \ --tests org.codehaus.groovy.vmplugin.v8.IndyScopedSwitchPointTest --rerun-tasks # Performance (HEAD) ./gradlew :perf:jmh -PbenchInclude=ScopedInvalidation -PjmhResultFormat=JSON ./gradlew :perf:jmh -PbenchInclude=CallSiteInvalidation -PjmhResultFormat=JSON # Performance (parent @ 0ca665dad0; copy ScopedInvalidationBench sources for fair A/B) # Run the same two :perf:jmh invocations in a clean worktree at the parent commit. ``` ## Appendix B — Commits between baseline and HEAD ```text c81e62bcca GROOVY-12191: Drop hierarchy SwitchPoint fan-out for stock metaclasses 637bdaf68a GROOVY-12191: Attach indy MOP SwitchPoints to MetaClass instances 3558689a06 GROOVY-12191: Document missing-method hierarchy and add blackdrag scenario tests 0303da79dc GROOVY-12191: MetaClass-aware SwitchPoint fan-out and PIC sentinel rename 1a31f276ca GROOVY-12191: Address blackdrag review on scoped SwitchPoint invalidation f48ab19256 GROOVY-12191: Address PR review and add index hierarchy for SwitchPoint fan-out ae8e49741e GROOVY-12191: Scope indy SwitchPoint invalidation ``` ## Appendix C — Key files (HEAD) - `org.apache.groovy.runtime.indy.IndyInvalidation` — MetaClass identity domains + exact/bulk policy - `org.apache.groovy.runtime.indy.SwitchPointInvalidator` - `ClassInfo` — pending domain + version coordination - `IndyInterface` / `Selector` / `ColdReflectiveMethodHandleWrapper` / `IndyCompoundAssign` — link-time guards - Tests: `Groovy12191`, `IndyInvalidationTest`, `SwitchPointInvalidatorTest`, `IndyScopedSwitchPointTest` - Benchmarks: `ScopedInvalidationBench`, `CallSiteInvalidationBench` ## Appendix D — Headline numbers (copy-ready) | Metric | Parent | HEAD | Gain | |---|---:|---:|---:| | Scoped unrelated thrpt | 0.00243 ops/ms | 0.512 ops/ms | **211×** | | CallSite crossType@1000 | 428 ms/op | 2.92 ms/op | **147×** | | CallSite burstThenSteady | 142 ms/op | 1.41 ms/op | **101×** | | Scoped parentChild parent-churn | 0.00242 ops/ms | 0.316 ops/ms | **130×** | | Scoped / CallSite baselines | — | — | **1.01–1.07×** (parity) | | Correctness | — | 94/94 | **green** | --- *Report generated from local JMH A/B measurements on 2026-08-01 (hostname `hera`, Corretto 25.0.2). Raw JSON and run logs: `/tmp/groovy-12191-perf-c81e62bcc/`.* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
