[
https://issues.apache.org/jira/browse/GROOVY-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16815914#comment-16815914
]
Jingfei Hu commented on GROOVY-9068:
------------------------------------
[~daniel_sun], Thank you very much for providing detailed workarounds. So based
on your response, I can see
# You're using GroovyShell instead of GroovyScriptEngine. That's the
difference and it matters.
# The method you declare it's not thread safe comes from
GroovyScriptEngineImpl actually, part of your library with version 2.5.6. So
according to your statements, GroovyScriptEngineImpl is not thread safe, I
doubt that.
# The following 2 solutions seem to work. However after I did a quick learning
about this topic here and there, the key point is a compromise between caching
Class objects parsed from scrip text and consumption of *Metaspace*
Below is the logic behind this, correct me if there is any misunderstanding.
* If we cache Class objects and never release them, then we would continuously
consume *Metaspace* and finally meet OOM as the number of different scripts
increases. In our application, that could occur in the near future.
* If we don't cache at all, which means we can meet below prerequisites after
we complete the evaluation, however it depends on when GC kicks in to reclaim
Class objects and the corresponding *Metaspace*, it's indeterministic and
Metaspace could be used up if there is a burst load and before GC actually does
its work. When we test it in out testbed environment, it confirmed the
suspicious, it quickly hit OOM merely after it's running one hour or so **
h3.
Prerequisites to unload the Class objects to release Metaspace
# There are no reachable instances of this Class
# The corresponding class loader object is GC-ed. So apparently sharing a
class loader object is against this rule
# There are no any references to this Class object, for instance there is no
way to access methods of this Class by reflection
So it seems to me that a cache which has a rotation/retiring mechanism function
is the "ultimate" solution, isn't it? Better ideas and suggestions are welcome!
> GroovyScriptEngine causes Metaspace OOM
> ---------------------------------------
>
> Key: GROOVY-9068
> URL: https://issues.apache.org/jira/browse/GROOVY-9068
> Project: Groovy
> Issue Type: Bug
> Components: GroovyScriptEngine
> Affects Versions: 2.4.9
> Environment: macOS Mojave, MacBook Pro (Retina, 15-inch, Mid 2015)
> Reporter: Jingfei Hu
> Priority: Major
> Labels: GroovyScriptEngineImpl, Metaspace, OOM
>
> Hello team,
> We've encountered this troublesome Metaspace OOM in our application recently
> as the number of groovy scripts increases. The groovy usage pattern in our
> application is evaluating scripts adhoc and directly. There are no any kinds
> of caches. And we use below code to do the evaluation.
>
> {code:java}
> engine.eval(scriptText, bindings);
> {code}
> We thought the cache of GroovyScriptEngineImpl which is below field would
> take effect, but actually not in the multi-threading context
> {code:java}
> // script-string-to-generated Class map
> private ManagedConcurrentValueMap<String, Class> classMap = new
> ManagedConcurrentValueMap<String, Class>(ReferenceBundle.getSoftBundle());
> {code}
> So without proper cache, our application continuously leverages the groovy
> class loader of GroovyScriptEngineImpl to parse scripts to generate Class
> objects and fill up Metaspace eventually.
>
> And below code snippets can easily reproduce this.
>
> {code:java}
> package com.jingfei;
> import java.util.concurrent.ExecutorService;
> import java.util.concurrent.Executors;
> public class Main {
> static final GroovyScriptExecutor groovyScriptExecutor = new
> GroovyScriptExecutor();
> public static void main(String[] args) throws InterruptedException {
> ExecutorService executor = Executors.newFixedThreadPool(10000);
> while (true) {
> executor.execute(new Runnable() {
> @Override
> public void run() {
> try {
> groovyScriptExecutor.executeScript("(1..10).sum()");
> } catch (Exception e) {
> e.printStackTrace();
> }
> }
> });
> }
> }
> }
> package com.jingfei;
> import java.util.HashMap;
> import java.util.Map;
> import javax.script.CompiledScript;
> import javax.script.ScriptEngine;
> import javax.script.ScriptEngineManager;
> import javax.script.ScriptException;
> import groovy.lang.GroovyClassLoader;
> import org.codehaus.groovy.jsr223.GroovyCompiledScript;
> import org.codehaus.groovy.jsr223.GroovyScriptEngineImpl;
> /**
> * @author jingfei
> * @version $Id: GroovyScriptExecutor.java, v 0.1 2019-04-02 20:07 jingfei
> Exp $$
> */
> public class GroovyScriptExecutor {
> /** Script Engine Manager */
> private static final ScriptEngineManager factory = new
> ScriptEngineManager();
> /** Script engine */
> private static final ScriptEngine engine =
> factory.getEngineByName("groovy");
> public void executeScript(final String scriptText) throws ScriptException
> {
> System.out.println(engine.eval(scriptText));
> }
> }{code}
> Looking into the Metaspace dump, we find out within the Metaspace there are
> hundreds of class loader objects named
> *groovy/lang/GroovyClassLoader$InnerLoader* and all of Class meta info loaded
> by them with the naming pattern *ScriptXXXX.*
>
> Since the script is all the same, i.e.
> {code:java}
> (1..10).sum(){code}
> , this behavior seems totally odd to me, because I thought due to the
> existence of the cache, there would be only one class loader created
> necessary to parse the script.
>
> Any help is appreciated to work out this problem! Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
