[
https://issues.apache.org/jira/browse/GROOVY-8388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16260056#comment-16260056
]
Shevek commented on GROOVY-8388:
--------------------------------
I don't know how to prevent creating all of those metaclasses.
Right now, I'm stuck for a workaround:
* I can't load, cache and clone a DelegatingScript for use in multiple threads.
* I don't have a good handle on what's making all the metaclasses, but saying
to a user "You may not do anything which creates a metaclass" might be a bit
much of a restriction.
* I'm about to try caching the returned Class object from GroovyShell and using
InvokerHelper.createScript myself.
* Can I make it do a linear time cleanup, possibly on the same thread?
Given that the only effective way to bulk-remove from a linked data structure
is to iterate, would a better implementation of
ManagedConcurrentLinkedQueue$Element.finalizeReference() to flag the queue
itself for a single cleanup, rather than iterating once per removed element?
It's not clear to me why the class set is growing so large; certainly I'm
putting through a lot of requests, and GC is not timely, but there's an
implication that either the references are getting enqueued not-in-queue-order
(unsurprising, you'd expect them to be enqueued tail-end-first, so they
automatically hit the worst possible case), or that they aren't getting
enqueued in a timely manner (unsurprising; aside from groovy, we tend to be
very low garbage).
> Groovy MetaClass manager hangs up for 2 minutes every 30 minutes
> ----------------------------------------------------------------
>
> Key: GROOVY-8388
> URL: https://issues.apache.org/jira/browse/GROOVY-8388
> Project: Groovy
> Issue Type: Bug
> Affects Versions: 2.4.12
> Environment: Linux x86, OpenJDK8
> Reporter: Shevek
>
> Groovy hangs on the host for over 2 minutes, at 30 minute intervals. Here is
> the stack:
> ```
> at
> java.util.concurrent.ConcurrentLinkedQueue.remove(ConcurrentLinkedQueue.java:490)
> at
> org.codehaus.groovy.util.ManagedConcurrentLinkedQueue$Element.finalizeReference(ManagedConcurrentLinkedQueue.java:117)
> at
> org.codehaus.groovy.util.ReferenceManager$CallBackedManager.removeStallEntries0(ReferenceManager.java:108)
> at
> org.codehaus.groovy.util.ReferenceManager$CallBackedManager.removeStallEntries(ReferenceManager.java:93)
> at
> org.codehaus.groovy.util.ReferenceManager$CallBackedManager.afterReferenceCreation(ReferenceManager.java:117)
> at
> org.codehaus.groovy.util.ReferenceManager$1.afterReferenceCreation(ReferenceManager.java:135)
> at
> org.codehaus.groovy.util.ManagedReference.<init>(ManagedReference.java:36)
> at
> org.codehaus.groovy.util.ManagedReference.<init>(ManagedReference.java:40)
> at
> org.codehaus.groovy.util.ManagedConcurrentLinkedQueue$Element.<init>(ManagedConcurrentLinkedQueue.java:112)
> at
> org.codehaus.groovy.util.ManagedConcurrentLinkedQueue.add(ManagedConcurrentLinkedQueue.java:62)
> at
> org.codehaus.groovy.reflection.ClassInfo$GlobalClassSet.add(ClassInfo.java:464)
> at
> org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:84)
> at
> org.codehaus.groovy.reflection.ClassInfo$1.computeValue(ClassInfo.java:80)
> at
> org.codehaus.groovy.reflection.GroovyClassValuePreJava7$EntryWithValue.<init>(GroovyClassValuePreJava7.java:37)
> at
> org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:64)
> at
> org.codehaus.groovy.reflection.GroovyClassValuePreJava7$GroovyClassValuePreJava7Segment.createEntry(GroovyClassValuePreJava7.java:55)
> at
> org.codehaus.groovy.util.AbstractConcurrentMap$Segment.put(AbstractConcurrentMap.java:157)
> at
> org.codehaus.groovy.util.AbstractConcurrentMap$Segment.getOrPut(AbstractConcurrentMap.java:100)
> at
> org.codehaus.groovy.util.AbstractConcurrentMap.getOrPut(AbstractConcurrentMap.java:38)
> at
> org.codehaus.groovy.reflection.GroovyClassValuePreJava7.get(GroovyClassValuePreJava7.java:94)
> at
> org.codehaus.groovy.reflection.ClassInfo.getClassInfo(ClassInfo.java:143)
> at
> org.codehaus.groovy.runtime.metaclass.MetaClassRegistryImpl.getMetaClass(MetaClassRegistryImpl.java:261)
> at
> org.codehaus.groovy.runtime.InvokerHelper.getMetaClass(InvokerHelper.java:883)
> at groovy.lang.GroovyObjectSupport.<init>(GroovyObjectSupport.java:34)
> at groovy.lang.Script.<init>(Script.java:42)
> at groovy.lang.Script.<init>(Script.java:39)
> at groovy.util.DelegatingScript.<init>(DelegatingScript.java:93)
> at hive-to-presto.<init>(hive-to-presto.groovy)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.codehaus.groovy.runtime.InvokerHelper.createScript(InvokerHelper.java:431)
> at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
> at groovy.lang.GroovyShell.parse(GroovyShell.java:709)
> [application code hereafter]
> ```
> Stack captured by stack sampling at 10s intervals on a failing application;
> stack remains consistent for the 2 minute period.
> I've noted on concurrency-interest that a couple of the j.u.c classes have
> slightly surprising (to us) complexity measures, partly because we're so used
> to j.u methods being very efficient; it's entirely possible that
> ConcurrentLinkedQueue.remove() takes linear time.
> History: Sometime around groovy 2.4.7 - fixed in groovy 2.4.8 - there was a
> bug where groovy never released references to classes, and would crash the
> JVM with something like 'Out of compressed OOPS'. The fix for this apparently
> introduced a walk of a queue, which now hangs up the JVM instead of breaking
> it.
> On groovy 2.4.7, the number of classes loaded before crash was about 238,000.
> We may be able to work around this by caching loaded classes, but that will
> impact the semantics of our application.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
