[
https://issues.apache.org/jira/browse/GROOVY-12332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18138823#comment-18138823
]
ASF GitHub Bot commented on GROOVY-12332:
-----------------------------------------
codecov-commenter commented on PR #3230:
URL: https://github.com/apache/groovy/pull/3230#issuecomment-6660559548
##
[Codecov](https://app.codecov.io/gh/apache/groovy/pull/3230?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
Report
:x: Patch coverage is `87.50000%` with `6 lines` in your changes missing
coverage. Please review.
:white_check_mark: Project coverage is 74.1750%. Comparing base
([`ee802a1`](https://app.codecov.io/gh/apache/groovy/commit/ee802a1814eae7a50016cf6ad2896ff9e80f6665?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache))
to head
([`8e3ed31`](https://app.codecov.io/gh/apache/groovy/commit/8e3ed31986121e4232dc35203d6e871170364b9b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing
lines](https://app.codecov.io/gh/apache/groovy/pull/3230?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
| Patch % | Lines |
|---|---|---|
|
[...va/org/apache/groovy/parser/antlr4/AstBuilder.java](https://app.codecov.io/gh/apache/groovy/pull/3230?src=pr&el=tree&filepath=src%2Fmain%2Fjava%2Forg%2Fapache%2Fgroovy%2Fparser%2Fantlr4%2FAstBuilder.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dyb292eS9wYXJzZXIvYW50bHI0L0FzdEJ1aWxkZXIuamF2YQ==)
| 84.6154% | [0 Missing and 6 partials :warning:
](https://app.codecov.io/gh/apache/groovy/pull/3230?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
|
<details><summary>Additional details and impacted files</summary>
[](https://app.codecov.io/gh/apache/groovy/pull/3230?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
```diff
@@ Coverage Diff @@
## master #3230 +/- ##
==================================================
+ Coverage 74.1723% 74.1750% +0.0028%
- Complexity 43760 43776 +16
==================================================
Files 1692 1692
Lines 149599 149646 +47
Branches 28386 28397 +11
==================================================
+ Hits 110961 111000 +39
- Misses 28430 28437 +7
- Partials 10208 10209 +1
```
| [Files with missing
lines](https://app.codecov.io/gh/apache/groovy/pull/3230?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
| Coverage Δ | |
|---|---|---|
|
[...g/apache/groovy/parser/antlr4/GroovyLangLexer.java](https://app.codecov.io/gh/apache/groovy/pull/3230?src=pr&el=tree&filepath=src%2Fmain%2Fjava%2Forg%2Fapache%2Fgroovy%2Fparser%2Fantlr4%2FGroovyLangLexer.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dyb292eS9wYXJzZXIvYW50bHI0L0dyb292eUxhbmdMZXhlci5qYXZh)
| `100.0000% <100.0000%> (ø)` | |
|
[.../apache/groovy/parser/antlr4/GroovyLangParser.java](https://app.codecov.io/gh/apache/groovy/pull/3230?src=pr&el=tree&filepath=src%2Fmain%2Fjava%2Forg%2Fapache%2Fgroovy%2Fparser%2Fantlr4%2FGroovyLangParser.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dyb292eS9wYXJzZXIvYW50bHI0L0dyb292eUxhbmdQYXJzZXIuamF2YQ==)
| `60.0000% <100.0000%> (+20.0000%)` | :arrow_up: |
|
[...va/org/apache/groovy/parser/antlr4/AstBuilder.java](https://app.codecov.io/gh/apache/groovy/pull/3230?src=pr&el=tree&filepath=src%2Fmain%2Fjava%2Forg%2Fapache%2Fgroovy%2Fparser%2Fantlr4%2FAstBuilder.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dyb292eS9wYXJzZXIvYW50bHI0L0FzdEJ1aWxkZXIuamF2YQ==)
| `87.4273% <84.6154%> (-0.0450%)` | :arrow_down: |
... and [18 files with indirect coverage
changes](https://app.codecov.io/gh/apache/groovy/pull/3230/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test
Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests,
report on failures, and find test suite problems.
- :package: [JS Bundle
Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save
yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
> A single large class OOMs at the default compiler heap since 6.0.0-beta-3
> -------------------------------------------------------------------------
>
> Key: GROOVY-12332
> URL: https://issues.apache.org/jira/browse/GROOVY-12332
> Project: Groovy
> Issue Type: Bug
> Reporter: Paul King
> Priority: Major
>
> h2. Summary
> Since 6.0.0-beta-3 the compiler needs materially more heap to compile a
> single very large class, enough to
> cross Gradle's default worker heap: the same sources that build on beta-3 die
> on a current snapshot with
> {{java.lang.OutOfMemoryError: Java heap space}} about fifteen seconds in.
> The useful part of the report is not the OOM but what does *not* OOM.
> Compiling MORE total source succeeds,
> provided it is spread over ordinary-sized classes. So this is not a leak
> proportional to input size; the cost
> tracks the size of the largest single class.
> h2. The measurement
> One project ([groovy-verify|https://github.com/paulk-asert/groovy-verify]),
> two source sets, same compiler,
> same 512m ceiling on a forked compiler JVM:
> || source set || total lines || largest class || 6.0.0-SNAPSHOT @ 512m ||
> | {{src/test}} | 36,218 | 555 | *compiles* |
> | {{src/main}} | 34,898 | *15,580* | *OOM* |
> The larger body of source is the one that compiles. {{src/main}}'s
> {{src/main/groovy/verification/VerifyChecker.groovy}} is a single 15,580-line
> class; nothing in
> {{src/test}} exceeds 555 lines.
> Across Groovy versions, {{src/main}} unchanged:
> || Groovy || 512m || 640m ||
> | 6.0.0-beta-3 | *OK* | OK |
> | 6.0.0-SNAPSHOT | *OOM* | OK |
> So the requirement moved from under 512m to about 640m for identical input —
> modest in itself, but it
> crosses the heap a default Gradle Groovy compile gets, which is why it
> presents as a hard build failure
> rather than as slower compilation.
> h2. Reproduction
> {code}
> git clone https://github.com/paulk-asert/groovy-verify && cd groovy-verify
> # with a 6.0.0 snapshot installed to ~/.m2
> cat > /tmp/h.init.gradle <<'EOF'
> allprojects {
> repositories { mavenLocal() }
> tasks.withType(GroovyCompile).configureEach {
> groovyOptions.fork = true
> groovyOptions.forkOptions.memoryMaximumSize = '512m'
> }
> }
> EOF
> ./gradlew compileGroovy -PgroovyVersion=6.0.0-SNAPSHOT --init-script
> /tmp/h.init.gradle # OOM
> ./gradlew compileGroovy -PgroovyVersion=6.0.0-beta-3 --init-script
> /tmp/h.init.gradle # passes
> {code}
> Raising the ceiling to {{640m}} makes the snapshot pass. Compiling the test
> source set at 512m (with the
> main set already built, since it is a dependency) also passes on the snapshot.
> Environment: JDK 25.0.3-tem, Gradle 9.7.1, macOS. At the default (unforked)
> worker heap the failure is
> reported as {{java.lang.OutOfMemoryError: Java heap space}} at
> {{org.gradle.api.internal.tasks.compile.ApiGroovyCompiler.execute}}; at very
> low ceilings it surfaces
> instead as a bare {{parsing failed}} with no file or line, which is worth
> knowing when triaging.
> h2. What it is not
> * *Not growth in the project being compiled.* An older commit of the same
> project — 34,107 lines of
> {{src/main}} against today's 34,898, i.e. 2.3% smaller — also OOMs at 512m
> on the snapshot. The smaller,
> older tree fails on the new compiler while the larger, current tree passes
> on beta-3.
> * *Not the parser DFA cache.* {{-Dgroovy.antlr4.cache.size=2000}}
> (GROOVY-12318) still OOMs at 512m. That
> same setting compiles the project fine at 6g, so the small cache is
> functionally sound and the 512m
> failure is memory elsewhere.
> * *Not GroovyClassValue retention.* Neither {{-Dgroovy.use.classvalue=soft}}
> (GROOVY-12281) nor
> {{-Dgroovy.use.classvalue=false}} changes the outcome at 512m.
> Every system property added since 6.0.0-beta-1 was enumerated and the
> memory-related ones tested:
> {{groovy.use.classvalue}}, {{groovy.antlr4.cache.size}},
> {{groovy.antlr4.cache.threshold}},
> {{groovy.errors.tolerance}}, {{groovy.indy.aot.link}},
> {{groovy.hidden.classes.disable}},
> {{groovy.grape.insecureProtocolPolicy}}. None of them addresses this.
> h2. Suspected cause — NOT confirmed
> GROOVY-12288 added {{CachingClassWriter}} in {{CompilationUnit}}, which
> memoises {{getCommonSuperClass}} per
> class:
> {code:java}
> private final Map<String, ClassNode> classNodeByInternalName = new
> HashMap<>(8);
> private final Map<String, Map<String, String>> commonSuperByPair = new
> HashMap<>(8);
> {code}
> {{commonSuperByPair}} is a map of maps over PAIRS of internal names, so it
> grows with the distinct type
> pairs met while writing one class, has no ceiling and no eviction, and lives
> for the whole of that class's
> generation. That shape — cost per class rather than per compilation unit,
> superlinear in the types inside a
> single class — is what the measurement above shows, and the commit post-dates
> beta-3. It also fits the
> absence of a knob: GROOVY-12288 introduced no property to bound or disable
> the memo.
> I have not confirmed this. Doing so needs a build of master with that commit
> reverted, compared at 512m;
> I did not want to overwrite the snapshot in the local repository to test it.
> If the theory holds, a bound on
> the memo (or clearing it per class rather than letting it live to the end of
> generation) should restore the
> earlier ceiling while keeping the speed-up the caching was added for.
> h2. Impact and workaround
> A project is affected in proportion to its largest class, not its size, so it
> will look arbitrary from the
> outside: most builds are unaffected and a few fail outright at the default
> heap. The workaround is to fork
> the Groovy compiler with a raised ceiling:
> {code:groovy}
> tasks.withType(GroovyCompile).configureEach {
> groovyOptions.fork = true
> groovyOptions.forkOptions.memoryMaximumSize = '2g'
> }
> {code}
> Worth resolving before final 6 release, since the failure mode gives no hint
> that class size is the
> variable.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
