[
https://issues.apache.org/jira/browse/GROOVY-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sterling Greene updated GROOVY-10772:
-------------------------------------
Description:
We're seeing this in a test with Gradle + Groovy 4, so it's hard to reproduce
exactly what we're seeing in pure Groovy.
In Gradle, we run a build with a 150MB byte array ("memory hog") added as a
property to a Project object.
https://github.com/gradle/gradle/blob/5c4e492a9dc1665e9a80235cc0fe9292ead88434/subprojects/workers/src/integTest/groovy/org/gradle/workers/internal/WorkerExecutorIntegrationTest.groovy#L228
We then run a second build and third build to show that this doesn't cause the
daemon to OOM by retaining old Project references.
With Groovy 3, this test passes. With Groovy 3 + indy jars + indy compilation,
this test passes.
With Groovy 4 (4.0.5), this test fails with a OOM.
https://ge.gradle.org/s/xywk2kx7iqdna/tests/:workers:embeddedIntegTest/org.gradle.workers.internal.WorkerExecutorIntegrationTest/does%20not%20leak%20project%20state%20across%20multiple%20builds?top-execution=1
I've seen failures with Java 8 and 11, but it _seems_ like the real problem is
easier to reproduce with Java 8.
Looking at the heap dump, we can see `CacheableCallSite` is involved somehow
with hanging on to the byte array. To make it easier to identified which
generation the byte array came from, I set the first byte to 1, 2, 3... In the
real failure, the OOM happens on the second invocation with the first
generation byte array held by `CacheableCallSite`.
This script produces a OOM with a similar looking GC-path:
class Project {
private final byte[] memoryHog = new byte[150*1024*1024]
}
def func = {
def project = new Project()
project.memoryHog[0] = 1
println "hello $it " + project
project = null
}
func(1)
func = {
def project = new Project()
project.memoryHog[0] = 2
println "hello $it " + project
project = null
}
func(2)
func = {
def project = new Project()
project.memoryHog[0] = 3
println "hello $it " + project
project = null
}
func(3)
I'm running this with:
> JAVA_OPTS="-Xmx512m -Xms256m -XX:+HeapDumpOnOutOfMemoryError" groovy
> memoryhog.groovy
This script runs with Groovy 3.0.12 with and without indy. This fails with
Groovy 4.0.4.
Would anything else be useful for diagnosing this?
Maybe related to https://issues.apache.org/jira/browse/GROOVY-10232
was:
We're seeing this in a test with Gradle + Groovy 4, so it's hard to reproduce
exactly what we're seeing in pure Groovy.
In Gradle, we run a build with a 150MB byte array ("memory hog") added as a
property to a Project object.
https://github.com/gradle/gradle/blob/5c4e492a9dc1665e9a80235cc0fe9292ead88434/subprojects/workers/src/integTest/groovy/org/gradle/workers/internal/WorkerExecutorIntegrationTest.groovy#L228
We then run a second build and third build to show that this doesn't cause the
daemon to OOM by retaining old Project references.
With Groovy 3, this test passes. With Groovy 3 + indy jars + indy compilation,
this test passes.
With Groovy 4 (4.0.5), this test fails with a OOM.
https://ge.gradle.org/s/xywk2kx7iqdna/tests/:workers:embeddedIntegTest/org.gradle.workers.internal.WorkerExecutorIntegrationTest/does%20not%20leak%20project%20state%20across%20multiple%20builds?top-execution=1
I've seen failures with Java 8 and 11, but it _seems_ like the real problem is
easier to reproduce with Java 8.
Looking at the heap dump, we can see `CacheableCallSite` is involved somehow
with hanging on to the byte array. To make it easier to identified which
generation the byte array came from, I set the first byte to 1, 2, 3... In the
real failure, the OOM happens on the second invocation with the first
generation byte array held by `CacheableCallSite`.
This script produces a OOM with a similar looking GC-path:
{{class Project {
private final byte[] memoryHog = new byte[150*1024*1024]
}
def func = {
def project = new Project()
project.memoryHog[0] = 1
println "hello $it " + project
project = null
}
func(1)
func = {
def project = new Project()
project.memoryHog[0] = 2
println "hello $it " + project
project = null
}
func(2)
func = {
def project = new Project()
project.memoryHog[0] = 3
println "hello $it " + project
project = null
}
func(3)}}
I'm running this with:
> JAVA_OPTS="-Xmx512m -Xms256m -XX:+HeapDumpOnOutOfMemoryError" groovy
> memoryhog.groovy
This script runs with Groovy 3.0.12 with and without indy. This fails with
Groovy 4.0.4.
Would anything else be useful for diagnosing this?
Maybe related to https://issues.apache.org/jira/browse/GROOVY-10232
> Possible memory leak, CacheableCallSite retains objects across invocations
> --------------------------------------------------------------------------
>
> Key: GROOVY-10772
> URL: https://issues.apache.org/jira/browse/GROOVY-10772
> Project: Groovy
> Issue Type: Bug
> Affects Versions: 4.0.4
> Reporter: Sterling Greene
> Priority: Major
> Attachments: real-problem-gc-paths.png, reproducer-gc-paths.png
>
>
> We're seeing this in a test with Gradle + Groovy 4, so it's hard to reproduce
> exactly what we're seeing in pure Groovy.
> In Gradle, we run a build with a 150MB byte array ("memory hog") added as a
> property to a Project object.
> https://github.com/gradle/gradle/blob/5c4e492a9dc1665e9a80235cc0fe9292ead88434/subprojects/workers/src/integTest/groovy/org/gradle/workers/internal/WorkerExecutorIntegrationTest.groovy#L228
> We then run a second build and third build to show that this doesn't cause
> the daemon to OOM by retaining old Project references.
> With Groovy 3, this test passes. With Groovy 3 + indy jars + indy
> compilation, this test passes.
> With Groovy 4 (4.0.5), this test fails with a OOM.
> https://ge.gradle.org/s/xywk2kx7iqdna/tests/:workers:embeddedIntegTest/org.gradle.workers.internal.WorkerExecutorIntegrationTest/does%20not%20leak%20project%20state%20across%20multiple%20builds?top-execution=1
> I've seen failures with Java 8 and 11, but it _seems_ like the real problem
> is easier to reproduce with Java 8.
> Looking at the heap dump, we can see `CacheableCallSite` is involved somehow
> with hanging on to the byte array. To make it easier to identified which
> generation the byte array came from, I set the first byte to 1, 2, 3... In
> the real failure, the OOM happens on the second invocation with the first
> generation byte array held by `CacheableCallSite`.
> This script produces a OOM with a similar looking GC-path:
> class Project {
> private final byte[] memoryHog = new byte[150*1024*1024]
> }
> def func = {
> def project = new Project()
> project.memoryHog[0] = 1
> println "hello $it " + project
> project = null
> }
> func(1)
> func = {
> def project = new Project()
> project.memoryHog[0] = 2
> println "hello $it " + project
> project = null
> }
> func(2)
> func = {
> def project = new Project()
> project.memoryHog[0] = 3
> println "hello $it " + project
> project = null
> }
> func(3)
> I'm running this with:
> > JAVA_OPTS="-Xmx512m -Xms256m -XX:+HeapDumpOnOutOfMemoryError" groovy
> > memoryhog.groovy
> This script runs with Groovy 3.0.12 with and without indy. This fails with
> Groovy 4.0.4.
> Would anything else be useful for diagnosing this?
> Maybe related to https://issues.apache.org/jira/browse/GROOVY-10232
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
