[
https://issues.apache.org/jira/browse/GROOVY-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885899#comment-15885899
]
Jesse Glick commented on GROOVY-8092:
-------------------------------------
bq. Did you guys do any analysis of the soft leak problem?
Sure, we have known about this for a long time. JENKINS-42189 shows how it
manifests itself in 2.4.8—you can [look yourself at the GC root reference
chain|https://issues.jenkins-ci.org/browse/JENKINS-42189?focusedCommentId=289094&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-289094]
printed by our automated test tool.
bq. I assume is that there is a bunch of SoftReferences not releasing their
Class typed referents, leading to the class loader not being unloaded.
Yes, the usual problem. Since Jenkins Pipeline (currently) creates a new
{{GroovyClassLoader}} per build, and every build loads at least one
({{Script}}) class which {{ClassInfo}} tracks, you get a substantial amount of
junk created with regularity. You can watch on a memory monitor or using a
simple script like
{code:none}
while :; do jmap -histo:live <<PID>> | fgrep GroovyClassLoader; sleep 10; done
{code}
while forcing Jenkins to run a build per second or so, and see the heap grow on
a steady slope.
bq. -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled could help here
No, there is a JVM flag to set some timeout for {{SoftReference}} to one second
or some such dummy value, which does help, but we cannot put the burden on
users to tune their JVM with unsupported flags. So Jenkins forcibly clears the
references when it is done with the {{GroovyClassLoader}}. [This is the current
code|https://github.com/jenkinsci/workflow-cps-plugin/blob/d59b417e08b84eb226f8074927d5fb653de5a662/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java#L943-L1087]
if you are interested.
Note that while Groovy is the worst offender in terms of leaks, there are other
faulty libraries used _by_ Groovy which also leak {{ClassLoader}} references,
namely {{ObjectStreamClass.Caches}} and {{java.beans.Introspector}}, as well as
{{SerializableClassRegistry}} in JBoss Marshalling which the Jenkins plugin
uses for other purposes. Without automated tests it would be impractical to
keep track of all these.
bq. {{ClassInfo}} could be changed to keep strong references to classes if it's
classloader is the same as or a child of the class' classloader (or if the
class' classloader is null), otherwise hold it weakly.
If I understand the proposal correctly, that would be great.
bq. a soft leak as I understood it is no memory leak per se. It is the GC not
doing its \[j\]ob properly.
The Java Platform makes no real guarantees about what a {{SoftReference}} will
do. As seen in our automated tests, in practice it will be collected only when
some allocation attempt is made which actually throws {{OutOfMemoryError}}. In
the meantime, they basically stick around indefinitely, causing the heap to
grow to its configured limit and stay very close to it, inhibiting normal GC
activity and making the whole application run very slowly. You can find plenty
of convincing articles from a quick Google search explaining why you should
never use {{SoftReference}} for anything. ({{WeakReference}} is perfectly fine.)
> Compatibility issues between groovy 2.4.8 and jenkins workflow-cps-plugin
> 2.23
> -------------------------------------------------------------------------------
>
> Key: GROOVY-8092
> URL: https://issues.apache.org/jira/browse/GROOVY-8092
> Project: Groovy
> Issue Type: Bug
> Components: groovy-runtime
> Affects Versions: 2.4.8
> Environment: Jenkins 2.19-stable with a single commit that bumps
> groovy 2.4.8
> workflow-cps-plugin 2.23
> Reporter: Ion Alberdi
>
> Created the issue on jenkins first
> https://issues.jenkins-ci.org/browse/JENKINS-42189
> Creating it here, in case the fix may be developed in groovy itself.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
