[
https://issues.apache.org/jira/browse/GROOVY-8388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16259735#comment-16259735
]
Shevek commented on GROOVY-8388:
--------------------------------
ConcurrentLinkedQueue.remove() does seem to be linear time, which makes it a
very bad choice for a synchronous callback which is invoked N times. So the 2
minutes is a quadratic-time cleanup operation blocking the working thread.
Other notes:
In createThresholdedIdlingManager(), it checks for a value against the
threshold but never resets the threshold? I don't actually understand what's
going on there - it's basically ignoring the first 5,000 enqueues?
Both ReferenceManager and ReferenceBundle have static members for weak and soft
ReferenceBundles.
There's no javadoc at all - is the point of the CallBackedManager to perform
the cleanup operations on the same thread?
This feels like as much spelunking as I can do right now; I recognize all the
patterns, but I'm not entirely sure WHY I'm dealing with a managed queue here
or quite what application behaviour causes growth in this queue. In any case,
for an application with a reasonable throughput of 'new' Groovy scripts, the
application shouldn't hang up like this.
Thank you.
> Groovy MetaClass manager hangs up for 2 minutes every 30 minutes
> ----------------------------------------------------------------
>
> Key: GROOVY-8388
> URL: https://issues.apache.org/jira/browse/GROOVY-8388
> Project: Groovy
> Issue Type: Bug
> Affects Versions: 2.4.12
> Environment: Linux x86, OpenJDK8
> Reporter: Shevek
>
> Groovy hangs on the host for over 2 minutes, at 30 minute intervals. Here is
> the stack:
> ```
> at
> java.util.concurrent.ConcurrentLinkedQueue.remove(ConcurrentLinkedQueue.java:490)
> at
> org.codehaus.groovy.util.ManagedConcurrentLinkedQueue$Element.finalizeReference(ManagedConcurrentLinkedQueue.java:117)
> at
> org.codehaus.groovy.util.ReferenceManager$CallBackedManager.removeStallEntries0(ReferenceManager.java:108)
> at
> org.codehaus.groovy.util.ReferenceManager$CallBackedManager.removeStallEntries(ReferenceManager.java:93)
> at
> org.codehaus.groovy.util.ReferenceManager$CallBackedManager.afterReferenceCreation(ReferenceManager.java:117)
> at
> org.codehaus.groovy.util.ReferenceManager$1.afterReferenceCreation(ReferenceManager.java:135)
> at
> org.codehaus.groovy.util.ManagedReference.<init>(ManagedReference.java:36)
> at
> org.codehaus.groovy.util.ManagedReference.<init>(ManagedReference.java:40)
> at
> org.codehaus.groovy.util.ManagedConcurrentLinkedQueue$Element.<init>(ManagedConcurrentLinkedQueue.java:112)
> at
> org.codehaus.groovy.util.ManagedConcurrentLinkedQueue.add(ManagedConcurrentLinkedQueue.java:62)
> at
> org.codehaus.groovy.reflection.ClassInfo$GlobalClassSet.add(ClassInfo.java:464)
> at
> org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:84)
> at
> org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:80)
> at
> org.codehaus.groovy.reflection.GroovyClassValuePreJava7$EntryWithValue.<init>(GroovyClassValuePreJava7.java:37)
> at
> org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:64)
> at
> org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:55)
> at
> org.codehaus.groovy.util.AbstractConcurrentMap$Segment.put(AbstractConcurrentMap.java:157)
> at
> org.codehaus.groovy.util.AbstractConcurrentMap$Segment.getOrPut(AbstractConcurrentMap.java:100)
> at
> org.codehaus.groovy.util.AbstractConcurrentMap.getOrPut(AbstractConcurrentMap.java:38)
> at
> org.codehaus.groovy.reflection.GroovyClassValuePreJava7.get(GroovyClassValuePreJava7.java:94)
> at
> org.codehaus.groovy.reflection.ClassInfo.getClassInfo(ClassInfo.java:143)
> at
> org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.getMetaClass(MetaClassRegistryImpl.java:261)
> at
> org.codehaus.groovy.runtime.InvokerHelper.getMetaClass(InvokerHelper.java:883)
> at groovy.lang.GroovyObjectSupport.<init>(GroovyObjectSupport.java:34)
> at groovy.lang.Script.<init>(Script.java:42)
> at groovy.lang.Script.<init>(Script.java:39)
> at groovy.util.DelegatingScript.<init>(DelegatingScript.java:93)
> at hive-to-presto.<init>(hive-to-presto.groovy)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.codehaus.groovy.runtime.InvokerHelper.createScript(InvokerHelper.java:431)
> at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
> at groovy.lang.GroovyShell.parse(GroovyShell.java:709)
> [application code hereafter]
> ```
> Stack captured by stack sampling at 10s intervals on a failing application;
> stack remains consistent for the 2 minute period.
> I've noted on concurrency-interest that a couple of the j.u.c classes have
> slightly surprising (to us) complexity measures, partly because we're so used
> to j.u methods being very efficient; it's entirely possible that
> ConcurrentLinkedQueue.remove() takes linear time.
> History: Sometime around groovy 2.4.7 - fixed in groovy 2.4.8 - there was a
> bug where groovy never released references to classes, and would crash the
> JVM with something like 'Out of compressed OOPS'. The fix for this apparently
> introduced a walk of a queue, which now hangs up the JVM instead of breaking
> it.
> On groovy 2.4.7, the number of classes loaded before crash was about 238,000.
> We may be able to work around this by caching loaded classes, but that will
> impact the semantics of our application.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
