daniellansun commented on PR #2736: URL: https://github.com/apache/groovy/pull/2736#issuecomment-5134195050
# GROOVY-12191 Performance Verification Report V3 **Scoped indy SwitchPoint invalidation (MetaClass identity-map domains)** | Item | Value | |---|---| | Issue | [GROOVY-12191](https://issues.apache.org/jira/browse/GROOVY-12191) | | Commit under test | `637bdaf68a2816930d4d45211584bc77bfe7a9b7` | | Baseline (parent) | `0ca665dad0d4070bfa218d9b5577ec423c8b2181` | | Date | 2026-07-31 | | Verdict | **PASS** — cross-type MetaClass churn no longer globally deoptimizes hot monomorphic call sites; steady-state hot path is parity; correctness suite green (107/107) | --- ## 1. Executive summary This verification compares the **full GROOVY-12191 stack** at `637bdaf68a` against the pre-scoping parent `0ca665dad0` (process-wide MOP `SwitchPoint`). At HEAD, indy MOP invalidation uses: - **One SwitchPoint domain per MetaClass instance** (weak identity map in `IndyInvalidation`). - **ClassInfo pending domain** only for pre-MetaClass link (defineClass safety); first MetaClass install retires it. - **MetaClass-aware fan-out**: EMC / modified mutable / interface / array / global EMC → class + hierarchy; pure unmodified `MetaClassImpl` replace → exact class only. - Linked call sites still carry **one** `SwitchPoint.guardWithTest` — the pre-6.0 monomorphic hot-path shape. On the same host, JDK, and JMH annotation settings, A/B measurement of **HEAD (`637bdaf68a`) vs baseline (`0ca665dad0`)** shows: 1. **Near-zero regression on steady-state monomorphic hot paths** (baseline ratios ≈ 0.98–1.13×, within noise). 2. **Large gains under cross-type MetaClass churn** — the primary win: - `CallSiteInvalidationBench.crossTypeInvalidationEvery1000`: **443.1 → 2.59 ms/op** (≈ **171×**). - `ScopedInvalidationBench.hotLoop_unrelatedMetaClassChurn`: throughput ≈ **151×**. 3. **Framework-style “unrelated burst then steady request loop”** (`burstThenSteadyState`): ≈ **112×**. 4. **Same-type invalidation and Category semantics are not “falsely optimized”**: same-type churn remains far slower than baseline (forced re-link); category enter/leave stays bulk and matches parent throughput (~1×). 5. **Correctness: 107 passed / 0 failed** (full scoped-invalidation suite on HEAD). > In one line: without adding a second hot-path guard, the change shrinks the blast radius of unrelated MetaClass mutations from process-global to MetaClass-/type-scoped (with correct hierarchy/array fan-out), with measured gains on the order of **10¹–10²×**. --- ## 2. Change mechanism and performance hypotheses ### 2.1 Previous model (baseline `0ca665dad0`) - `IndyInterface.switchPoint` was a **process-wide shared** `SwitchPoint`. - Any MetaClass registry change → `invalidateSwitchPoints()` → **every** linked indy site’s guard failed → mass re-link. - Typical symptom: a framework/plugin extends MetaClass on *ColdType* at startup or hot-reload; application hot paths on *HotTarget.compute()* were deoptimized together with it. ```text // baseline (simplified) protected static SwitchPoint switchPoint = new SwitchPoint(); // MetaClassRegistry listener → always: synchronized (IndyInterface.class) { SwitchPoint old = switchPoint; switchPoint = new SwitchPoint(); SwitchPoint.invalidateAll(new SwitchPoint[]{old}); } ``` ### 2.2 New model (`637bdaf68a`) | Component | Role | |---|---| | `SwitchPointInvalidator` | Single-domain lifecycle: CAS `get` / `detachLive` / `invalidate` | | `IndyInvalidation` | Policy + weak identity map `MetaClass → SwitchPointInvalidator`; hierarchy / category / exact-class | | `ClassInfo` | Pending domain for pre-MetaClass link only; version bumps; hierarchy index registration | | `ClassHierarchyIndex` | Weak ancestor→descendant index; O(\|subtypes\|) fan-out incl. JLS array lattice | | `Selector` / `IndyCompoundAssign` / `ColdReflectiveMethodHandleWrapper` | Install MetaClass-domain (or pending) SwitchPoint guard at link time | **Hot-path shape (intentionally monomorphic):** ```text handle = mopSwitchPoint.guardWithTest(fast, fallback) ``` Still a single guard—matching the pre-6.0 monomorphic shape. There is **no** second category SwitchPoint on the hot path. **Invalidation policy:** | Event | Behavior | Expected hot-path cost | |---|---|---| | Unrelated-type MetaClass change | Retire that MetaClass domain (and indexed subtypes when fan-out applies) only | Hot sites **do not** re-link | | Same-type / parent EMC MetaClass change | Retire class + hierarchy fan-out | Must re-link (correctness) | | Pure unmodified `MetaClassImpl` class replace | Exact class only | Subtypes not force-relinked | | Category enter/leave, `invalidateCallSites()` | Bulk-retire class-level / pending domains of loaded types | Same order of cost as old global invalidation | | Unattributed MetaClass event | `invalidateUnscoped()` bulk | Conservative, correctness-first | ### 2.3 Hypotheses under test | ID | Hypothesis | How verified | |---|---|---| | H1 | Baseline hot loop has no material regression | A/B ratio ≈ 1 | | H2 | Cross-type churn throughput/latency improves substantially | Unrelated ≫ parent; on HEAD, unrelated ≫ same-type | | H3 | Pure hot loop after unrelated burst approaches baseline | `afterUnrelatedBurst` ≈ `baseline` | | H4 | Same-type / hierarchy / category still pay re-link tax | Clearly slower than baseline | | H5 | Correctness suite stays green | Unit / integration tests | --- ## 3. Methodology ### 3.1 Environment | Item | Value | |---|---| | OS | Linux 6.15.5 x86_64 (hostname: `hera`) | | CPU | AMD EPYC 7763 (6 vCPUs visible) | | Memory | 23 GiB | | JDK | Amazon Corretto **25.0.2** (`25.0.2+10-LTS`) | | Build | Gradle wrapper, `./gradlew :perf:jmh`, indy enabled by default | | Trees | HEAD worktree `@ 637bdaf68a`; baseline worktree `@ 0ca665dad0` | | Bench source parity | `ScopedInvalidationBench` copied into parent worktree for fair A/B (not present on parent history); `CallSiteInvalidationBench` workloads identical | | Run order | Sequential: HEAD scoped → HEAD callsite → parent scoped → parent callsite (avoid dual-JVM core contention); no CPU pinning | ### 3.2 Benchmark suites 1. **`org.apache.groovy.bench.ScopedInvalidationBench`** (Throughput, ops/ms) - 100 000 monomorphic calls per op; MetaClass write every 1000 iterations in churn cases. - JMH: `@Warmup(3×2s) @Measurement(5×2s) @Fork(2)` → 10 samples per benchmark. 2. **`org.apache.groovy.perf.grails.CallSiteInvalidationBench`** (AverageTime, ms/op) - Same 100 000-iteration loops; cross-type / same-type / burst patterns. - Same JMH annotation settings. ### 3.3 Correctness ```bash ./gradlew :test --tests Groovy12191 \ --tests org.apache.groovy.runtime.indy.IndyInvalidationTest \ --tests org.apache.groovy.runtime.indy.SwitchPointInvalidatorTest \ --tests org.apache.groovy.runtime.indy.ClassHierarchyIndexTest \ --tests org.codehaus.groovy.vmplugin.v8.IndyScopedSwitchPointTest --rerun-tasks ``` **Result (HEAD `637bdaf68a`): 107 passed / 0 failed / 0 skipped / 0 error.** | Test class | Cases | |---|---:| | `bugs.Groovy12191` | 24 | | `IndyInvalidationTest` | 47 | | `SwitchPointInvalidatorTest` | 12 | | `ClassHierarchyIndexTest` | 12 | | `IndyScopedSwitchPointTest` | 12 | | **Total** | **107** | ### 3.4 Statistics and artifacts - Scores are JMH means; `±` is JMH’s default **99.9% CI**. - **Throughput** ratio = HEAD / parent (>1 is better). - **AverageTime** speedup = parent / HEAD (>1 is better). - Conclusions rest on **order-of-magnitude and structural separation**, not ±1% micro-deltas. Raw JSON (local run artifacts, not committed): - `/tmp/groovy-12191-perf-637bdaf/head/scoped-results.json` - `/tmp/groovy-12191-perf-637bdaf/parent/scoped-results.json` - `/tmp/groovy-12191-perf-637bdaf/head/callsite-results.json` - `/tmp/groovy-12191-perf-637bdaf/parent/callsite-results.json` - `/tmp/groovy-12191-perf-637bdaf/summary.json` --- ## 4. Results ### 4.1 ScopedInvalidationBench (Throughput, ops/ms — higher is better) | Benchmark | Parent | HEAD | HEAD/Parent | Interpretation | |---|---:|---:|---:|---| | `hotLoop_baseline` | 3.756 ± 0.151 | 4.015 ± 0.274 | **1.07×** | H1: no hot-path regression | | `hotLoop_afterUnrelatedBurst` | 3.693 ± 0.439 | 4.352 ± 0.423 | **1.18×** | H3: full speed after unrelated burst | | `hotLoop_unrelatedMetaClassChurn` | 0.002336 ± 0.000187 | 0.352 ± 0.064 | **≈151×** | **H2 primary win** | | `hotLoop_sameTypeMetaClassChurn` | 0.002334 ± 0.000234 | 0.0195 ± 0.0013 | **≈8.4×** | Still forced re-link; narrower blast radius than parent | | `hotLoop_categoryEnterLeave` | 0.00221 ± 0.00026 | 0.00209 ± 0.00045 | **≈0.95×** | H4: bulk semantics retained (parity within noise) | | `hierarchy_baseline` | 3.133 ± 0.122 | 2.817 ± 0.103 | 0.90× | Both full-speed order; container noise | | `hierarchy_parentMetaClassChurn` | 0.00217 ± 0.00025 | 0.00929 ± 0.00066 | **≈4.3×** | H4: parent change still invalidates child sites | **Structural fingerprint on HEAD only:** ```text baseline ≈ 4.01 ops/ms after burst ≈ 4.35 ops/ms (≥ baseline: no residual deopt) unrelated churn ≈ 0.35 ops/ms (~11× slower than baseline: mostly MetaClass write cost) same-type churn ≈ 0.019 ops/ms (~210× slower than baseline: write + same-class re-link) category ≈ 0.002 ops/ms (bulk re-link tax) ``` On parent, `unrelated` and `same-type` **both collapse to ~0.002**, showing that the old global SwitchPoint made “unrelated churn” equivalent to “global deopt.” HEAD separates them by ~**18×** (0.352 / 0.019)—a direct fingerprint that scoping is live. ### 4.2 CallSiteInvalidationBench (AverageTime, ms/op — lower is better) | Benchmark | Parent | HEAD | Speedup (P/H) | Interpretation | |---|---:|---:|---:|---| | `baselineHotLoop` | 0.262 ± 0.016 | 0.253 ± 0.027 | **1.03×** | H1 | | `baselineListSize` | 0.242 ± 0.010 | 0.246 ± 0.024 | **0.98×** | H1 (noise) | | `baselineSteadyStateNoBurst` | 0.630 ± 0.052 | 0.593 ± 0.053 | **1.06×** | H1 | | `baselineMultipleCallSites` | 21.56 ± 1.39 | 19.03 ± 1.57 | **1.13×** | H1 | | `crossTypeInvalidationEvery10000` | 321.5 ± 36.3 | 2.689 ± 0.678 | **≈120×** | H2 | | `crossTypeInvalidationEvery1000` | 443.1 ± 29.3 | 2.594 ± 1.353 | **≈171×** | **H2 primary win** | | `crossTypeInvalidationEvery100` | 148.7 ± 18.8 | 8.123 ± 1.167 | **≈18×** | Still large under denser churn | | `listSizeWithCrossTypeInvalidation` | 354.1 ± 50.6 | 1.738 ± 0.315 | **≈204×** | JDK-type hot sites benefit too | | `multipleCallSitesWithInvalidation` | 1214 ± 181 | 25.34 ± 2.19 | **≈48×** | Multi-site still much improved | | `burstThenSteadyState` | 159.7 ± 19.3 | 1.432 ± 0.104 | **≈112×** | “Framework MC extend + request steady state” | | `sameTypeInvalidationEvery1000` | 422.8 ± 20.0 | 54.46 ± 4.38 | **≈7.8×** | Same-type still slow; local invalidation cheaper than global rotate | **Penalty vs baseline on HEAD:** | Scenario | Relative cost | Meaning | |---|---:|---| | crossType @1000 | 2.59 / 0.25 ≈ **10×** `baselineHotLoop` | Dominated by ~100 ColdType MetaClass writes, not hot-site deopt | | sameType @1000 | 54.5 / 0.25 ≈ **215×** `baselineHotLoop` | Clear re-link tax | | burstThenSteady | 1.43 / 0.59 ≈ **2.4×** steady baseline | Burst write cost dominates; steady calls recovered | On parent, crossType@1000 and sameType@1000 are **both ~430 ms/op**, again confirming global invalidation collapsed cross-type into full deopt. --- ## 5. Why it is faster ```text Old path New path ColdType.metaClass.foo = ... ColdType.metaClass.foo = ... │ │ ▼ ▼ invalidate global SwitchPoint retire MetaClass(ColdType) domain │ (+ hierarchy only if policy requires) ▼ │ ALL sites: guard fails ▼ HotTarget.compute → full re-select ColdType sites only (MH rebuild / re-guard / cold path) HotTarget.compute → stays linked (single SwitchPoint check passes) ``` Cost breakdown: 1. **Avoided re-links** — After each global invalidation, hot sites re-run `Selector`, reinstall guards, and rewrite `CallSite`s. At 100 churn events × many sites this dominates (parent’s 400+ ms/op). 2. **Avoided JIT churn** — Frequent process-wide `SwitchPoint.invalidateAll` breaks monomorphic inlining assumptions and amplifies deopt/recompile cost. 3. **Unchanged hot-path guard count** — Still one `guardWithTest`, so baselines stay flat (H1). 4. **Same-type / hierarchy still correctly charged** — Prevents “fast but wrong” semantics (H4 + 107 tests). 5. **Category remains bulk** — No second hot-path guard; category visibility stays simple and correct; cost matches the old global path (measured ~1.0×). Same-type still improves ~**8×** on HEAD vs parent, not because re-link is skipped, but because the invalidation surface shrinks from “all process sites” to “this class hierarchy” and batch `invalidateAll` is cheaper—a secondary benefit, not the primary goal. --- ## 6. Risks and boundaries | Topic | Assessment | |---|---| | Hot-path regression | **Not observed** (baselines 0.98–1.13×) | | Semantic regression | 107/107 scoped suite green; Category / hierarchy / per-instance / MetaClass identity domain covered | | Dense Category enter/leave | **Still expensive** (by design); Category-heavy workloads do not get faster | | Parent EMC / hierarchy | Fan-out still paid for missing-method correctness; pure `MetaClassImpl` replace is exact-class | | Legacy `IndyInterface.switchPoint` | **Removed** on this branch (`vmplugin` internal-by-intent); external code must use `IndyInvalidation` | | Environment noise | Shared/container CPUs; conclusions use order-of-magnitude gaps (10¹–10²×), robust to ±10% noise | | Not covered here | Concurrent multi-thread churn throughput, full Grails end-to-end, JDK 17/21 matrix (recommended for CI) | --- ## 7. Hypothesis scorecard | Hypothesis | Result | Evidence | |---|---|---| | H1 baseline parity | **Holds** | Scoped baseline 1.07×; CallSite baselines 0.98–1.13× | | H2 cross-type speedup | **Holds** | ≈151× thrpt; ≈171× avgt (@1000) | | H3 post-unrelated-burst full speed | **Holds** | `afterUnrelatedBurst` ≥ baseline on HEAD | | H4 necessary invalidation still paid | **Holds** | same-type / hierarchy / category ≪ baseline | | H5 correctness | **Holds** | 107/107 tests | --- ## 8. Conclusions and recommendations ### Verdict `637bdaf68a` ([GROOVY-12191](https://issues.apache.org/jira/browse/GROOVY-12191)) keeps a **single-guard monomorphic hot path** while scoping indy MOP invalidation to **per-MetaClass domains** (+ hierarchy fan-out where MOP visibility requires it). For **unrelated-type MetaClass churn with hot monomorphic call sites**—a Grails/framework-shaped load—measured improvement is on the order of **10¹–10²×**. Steady-state undisturbed paths show **no material regression**. Correctness costs for Category and same-type changes **remain visible and intentional**. **Performance verification: PASS.** ### Recommendations 1. Keep `ScopedInvalidationBench` and the cross-type / burst rows of `CallSiteInvalidationBench` in regular JMH regression so a return to a global SwitchPoint cannot land silently. 2. In release notes, state clearly that Category and unscoped MetaClass events may still bulk-invalidate; the win is **type-/MetaClass-local MetaClass change**. 3. Optionally re-run CallSite `crossType@1000` and baselines on JDK 17/21 to confirm cross-LTS consistency. 4. Document that MOP guards are resolved via `IndyInvalidation` (MetaClass identity map + ClassInfo pending), not a process-wide `IndyInterface.switchPoint`. --- ## Appendix A — Reproduction commands ```bash # Correctness (HEAD @ 637bdaf68a) ./gradlew :test --tests Groovy12191 \ --tests org.apache.groovy.runtime.indy.IndyInvalidationTest \ --tests org.apache.groovy.runtime.indy.SwitchPointInvalidatorTest \ --tests org.apache.groovy.runtime.indy.ClassHierarchyIndexTest \ --tests org.codehaus.groovy.vmplugin.v8.IndyScopedSwitchPointTest --rerun-tasks # Performance (HEAD) ./gradlew :perf:jmh -PbenchInclude=ScopedInvalidation -PjmhResultFormat=JSON ./gradlew :perf:jmh -PbenchInclude=CallSiteInvalidation -PjmhResultFormat=JSON # Performance (parent @ 0ca665dad0; copy ScopedInvalidationBench sources for fair A/B) # Run the same two :perf:jmh invocations in a clean worktree at the parent commit. ``` ## Appendix B — Commits between baseline and HEAD ```text 637bdaf68a GROOVY-12191: Attach indy MOP SwitchPoints to MetaClass instances 3558689a06 GROOVY-12191: Document missing-method hierarchy and add blackdrag scenario tests 0303da79dc GROOVY-12191: MetaClass-aware SwitchPoint fan-out and PIC sentinel rename 1a31f276ca GROOVY-12191: Address blackdrag review on scoped SwitchPoint invalidation f48ab19256 GROOVY-12191: Address PR review and add index hierarchy for SwitchPoint fan-out ae8e49741e GROOVY-12191: Scope indy SwitchPoint invalidation ``` ## Appendix C — Key files (HEAD) - `org.apache.groovy.runtime.indy.IndyInvalidation` — MetaClass identity domains + policy - `org.apache.groovy.runtime.indy.SwitchPointInvalidator`, `ClassHierarchyIndex` - `ClassInfo` — pending domain + version coordination - `IndyInterface` / `Selector` / `ColdReflectiveMethodHandleWrapper` / `IndyCompoundAssign` — link-time guards - Tests: `Groovy12191`, `IndyInvalidationTest`, `SwitchPointInvalidatorTest`, `ClassHierarchyIndexTest`, `IndyScopedSwitchPointTest` - Benchmarks: `ScopedInvalidationBench`, `CallSiteInvalidationBench` --- *Report generated from local JMH A/B measurements on 2026-07-31 (hostname `hera`, Corretto 25.0.2). Raw JSON and run logs: `/tmp/groovy-12191-perf-637bdaf/`.* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
