[ https://issues.apache.org/jira/browse/GROOVY-10307 ]
Paul King deleted comment on GROOVY-10307:
------------------------------------
was (Author: paulk):
Here is ChatGPT's analysis of class vs indy performance using the bench and
perf benchmarks (prior to James' PR being applied):
{noformat}
1. Dispatch / Throughput (unchanged conclusion)
Indy=true still clearly improves dynamic dispatch throughput:
Monomorphic: ~+55%
3-way polymorphic: ~+45%
8-way megamorphic: ~+79%
hashCode: ~+48%
GroovyCS and Java remain largely unaffected by Indy.
Conclusion: Indy is beneficial for dynamic callsite throughput.
2. Recursion (Ackermann / Fibonacci)
Still consistent:
Indy=true is ~1.6–2.5× slower than Indy=false for recursive workloads.
Java unchanged.
Conclusion: Indy hurts deep recursion.
3. AryBench (Now Including groovyCS)
This is where the new data is very important.
Results at 1,000,000 elements:
Variant Time (ms)
Groovy (Indy=true) 21011
Groovy (Indy=false) 84
GroovyCS (@CompileStatic) 85
Java 84
Key Observations
GroovyCS matches Java almost exactly.
~85 ms vs Java’s ~84 ms
Essentially no abstraction penalty.
Groovy (Indy=false) also matches Java.
~84 ms
Groovy (Indy=true) is ~250× slower.
This is a dramatic regression.
Interpretation
The AryBench regression is not a “Groovy vs Java” issue.
It is specifically:
Dynamic Groovy + Indy in tight array-heavy loops.
Because:
Turning off Indy fixes it.
Using @CompileStatic fixes it.
Java baseline is fine.
Conclusion:
The AryBench anomaly is clearly tied to Indy dynamic dispatch in tight loops,
not Groovy itself.
This strongly suggests a callsite / invokedynamic interaction that prevents JIT
optimization in this workload.
4. Closures
Still mostly positive for Indy:
Many closure operations are ~1.5×–3× faster with Indy.
closureSpread remains a large regression with Indy (~20× slower).
GroovyCS generally performs closer to Java and avoids dynamic overhead.
Conclusion:
Indy helps most closure scenarios but has isolated pathological cases.
5. Method Invocation
Mixed but trending positive:
Simple/dynamic calls → faster with Indy.
Polymorphic callsite microbench → significantly slower with Indy (~7×).
GroovyCS again removes most dynamic overhead.
6. Property / Operators / Idioms
Mixed bag:
Some property access faster with Indy.
Field read/write slower with Indy.
Certain operators regress, others improve.
No single consistent trend.
GroovyCS generally stabilizes performance closer to Java.
Updated Executive Summary
What Indy Clearly Improves
Dynamic callsite throughput (45–80% gains)
Most closure-heavy workloads
Many dynamic method invocation scenarios
What Indy Clearly Hurts
Deep recursion (~1.6–2.5× slower)
Tight numeric/array loops (massive regression in AryBench)
Certain polymorphic microbenchmarks
closureSpread
Some field/operator microbenches
What groovyCS Shows
The new AryBench data proves:
Groovy itself can match Java performance.
Indy=false dynamic Groovy can also match Java here.
The large regression is specifically dynamic + Indy interaction.
@CompileStatic completely eliminates the issue.
High-Level Takeaways
Indy is a win for typical dynamic application workloads.
Indy can severely degrade tight numeric/array-heavy loops.
@CompileStatic restores Java-level performance.
The AryBench result strongly suggests:
An optimization barrier introduced by invokedynamic
Possibly preventing JIT loop optimizations or inlining
If prioritizing real-world app performance → Indy is mostly beneficial.
If targeting numeric/algorithmic hot loops →
Use @CompileStatic, or consider Indy=false until the regression is understood.
{noformat}
I'll relook at the impacts of the PR next...
> Groovy 4 runtime performance on average 2.4x slower than Groovy 3
> -----------------------------------------------------------------
>
> Key: GROOVY-10307
> URL: https://issues.apache.org/jira/browse/GROOVY-10307
> Project: Groovy
> Issue Type: Bug
> Components: bytecode, performance
> Affects Versions: 4.0.0-beta-1, 3.0.9
> Environment: OpenJDK Runtime Environment AdoptOpenJDK-11.0.11+9
> (build 11.0.11+9)
> OpenJDK 64-Bit Server VM AdoptOpenJDK-11.0.11+9 (build 11.0.11+9, mixed mode)
> WIN10 (tests) / REL 8 (web application)
> IntelliJ 2021.2
> Reporter: mgroovy
> Priority: Major
> Attachments: groovy_3_0_9_gc.png, groovy_3_0_9_loop2.png,
> groovy_3_0_9_loop4.png, groovy_3_0_9_mem.png, groovy_4_0_0_b1_loop2.png,
> groovy_4_0_0_b1_loop4.png, groovy_4_0_0_b1_loop4_gc.png,
> groovy_4_0_0_b1_loop4_mem.png,
> groovysql_performance_groovy4_2_xx_yy_zzzz.groovy, loops.groovy,
> profile3.txt, profile4-loops.txt, profile4.txt, profile4d.txt
>
>
> Groovy 4.0.0-beta-1 runtime performance in our framework is on average 2 to 3
> times slower compared to using Groovy 3.0.9 (regular i.e. non-INDY)
> * Our complete framework and application code is completely written in
> Groovy, spread over multiple IntelliJ modules
> ** mixed @CompileDynamic/@TypeChecked and @CompileStatic
> ** No Java classes left in project, i.e. no cross compilation occurs
> * We build using IntelliJ 2021.2 Groovy build process, then run / deploy the
> compiled class files
> ** We do _not_ use a Groovy based DSL, nor do we execute Groovy scripts
> during execution
> * Performance degradation when using Groovy 4.0.0-beta-1 instead of Groovy
> 3.0.9 (non-INDY):
> ** The performance of the largest of our web applications has dropped 3x
> (startup) / 2x (table refresh) respectively
> *** Stack: Tomcat/Vaadin/Ebean plus framework generated SQL
> ** Our test suite runs about 2.4 times as long as before (120 min when using
> G4, compared to about 50 min with G3)
> *** JUnit 5
> *** test suite also contains no scripts / dynamic code execution
> *** Individual test performance varies: A small number of tests runs faster,
> but the majority is slower, with some extreme cases taking nearly 10x as long
> to finish
> * Using Groovy 3.0.9 INDY displays nearly identical performance degradation,
> so it seems that the use of invoke dynamic is somehow at fault
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
