Hi there,
In the Java unit test listed further below, I create 100 GroovyShell
instances and add the same directory to the classpath of the
GroovyClassLoader of each GroovyShell. This directory contains 100
script files, named Util0.groovy to Util99.groovy, containing the
following script (with XX replaced by 0..99):
@Grab('com.google.guava:guava:18.0')
import com.google.common.base.Ascii
class UtilXX {
static boolean isUpperCase(def c) {
return Ascii.isUpperCase(c)\n"
}
}
The unit test then runs
shell.evaluate("return Util" + j + ".isUpperCase('C' as char)");
in 100 separate threads (with j from 0..99).
This test does not always fail, but often. Most of the time, a
ConcurrentModificationException occurs down in ivy (which is used by
Grape). In other cases, a MultipleCompilationErrorsException ocurred at
"@Grab('com.google.guava:guava:18.0')". (Full stacktraces at the bottom.)
It looks to me like Grape resp. ivy is not thread safe.
Question 1: Is that a bug or just how things currently are? (If it's a
bug, rather Grape bug or an ivy bug, resp. where would I best file it?)
Question 2: If it's not a bug, is that a general issue, i.e. if I
compile two sets of sources with the GroovyCompiler (say I create a
CompilationUnit instance and add sources) will there be similar issues
if both sets of sources have the same Grape dependencies? (Yes, I know,
I could just try, but maybe the answer obvious to someone who knows the
implementation.)
(I have tested this with Groovy 2.4.3 and ivy 2.4.0 on MacOS X with Java
6, but also I have also seen the same issues on CentOS with several
Groovy/ivy versions and Java 7, on a deployed webapp, so it seems to be
fairly independent of the exact environment.)
Best wishes,
Alain
-------------------------------------
Java Unit Test (does not always fail)
-------------------------------------
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import groovy.lang.GroovyShell;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.PrintWriter;
import java.io.UnsupportedEncodingException;
import java.util.LinkedList;
import java.util.List;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;
public class GrapeAndGroovyShellConcurrencyTest {
@Rule
public TemporaryFolder tempFolder = new TemporaryFolder();
private volatile boolean testPassed;
@Before
public void setUp() {
testPassed = true;
}
public static void setFileText(File file, String text)
throws FileNotFoundException, UnsupportedEncodingException {
PrintWriter writer = new PrintWriter(file, "UTF-8");
writer.write(text);
writer.close();
}
@Test
public void testConcurrency() throws Exception {
File dir = tempFolder.getRoot();
final int n = 100;
for (int i=0; i<n; i++) {
File f1 = new File(dir, "Util" + i + ".groovy");
setFileText(f1, "@Grab('com.google.guava:guava:18.0')\n"
+ "import com.google.common.base.Ascii\n"
+ "class Util" + i + " {\n"
+ " static boolean isUpperCase(def c) {\n"
+ " return Ascii.isUpperCase(c)\n"
+ " }\n"
+ "}\n"
);
}
List<Thread> scriptThreads = new LinkedList<Thread>();
for (int i=0; i<n; i++) {
final GroovyShell shell = new GroovyShell();
shell.getClassLoader().addClasspath(dir.getAbsolutePath());
final int j = i;
Thread scriptThread = new Thread(
new Runnable() {
public void run() {
try {
assertEquals(true,
shell.evaluate("return Util" + j + ".isUpperCase('C' as char)"));
System.out.println("Thread " +
Thread.currentThread().getName() + " : OK");
} catch (Throwable t) {
System.out.println("Thread " +
Thread.currentThread().getName() + " : FAILED");
t.printStackTrace();
testPassed = false;
}
}
});
scriptThread.setDaemon(true);
scriptThread.setName("run-" + i);
scriptThread.start();
scriptThreads.add(scriptThread);
}
for (Thread scriptThread : scriptThreads) {
scriptThread.join();
}
assertTrue("must be true", testPassed);
}
}
------------
Stacktrace 1
------------
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup
failed:
General error during conversion: java.util.ConcurrentModificationException
java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859)
at java.util.ArrayList$Itr.next(ArrayList.java:831)
at
org.apache.ivy.util.MessageLoggerHelper.sumupProblems(MessageLoggerHelper.java:45)
at
org.apache.ivy.util.MessageLoggerEngine.sumupProblems(MessageLoggerEngine.java:136)
at org.apache.ivy.util.Message.sumupProblems(Message.java:143)
at
org.apache.ivy.core.resolve.ResolveEngine.resolve(ResolveEngine.java:347)
at org.apache.ivy.Ivy.resolve(Ivy.java:523)
at org.apache.ivy.Ivy$resolve$1.call(Unknown Source)
at groovy.grape.GrapeIvy.getDependencies(GrapeIvy.groovy:404)
at sun.reflect.GeneratedMethodAccessor671.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSite.invoke(PogoMetaMethodSite.java:166)
at
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:56)
at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:563)
at groovy.grape.GrapeIvy$resolve$56.callCurrent(Unknown Source)
at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:532)
at groovy.grape.GrapeIvy$resolve$45.callCurrent(Unknown Source)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:151)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:179)
at groovy.grape.GrapeIvy.grab(GrapeIvy.groovy:254)
at groovy.grape.Grape.grab(Grape.java:163)
at
groovy.grape.GrabAnnotationTransformation.visit(GrabAnnotationTransformation.java:358)
at
org.codehaus.groovy.transform.ASTTransformationVisitor$3.call(ASTTransformationVisitor.java:319)
at
org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:928)
at
org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:590)
at
org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:566)
at
org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:543)
at
groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:297)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:267)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:692)
at groovy.lang.GroovyShell.parse(GroovyShell.java:704)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:588)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:627)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:598)
at
GrapeAndGroovyShellConcurrencyTest$1.run(GrapeAndGroovyShellConcurrencyTest.java:64)
at java.lang.Thread.run(Thread.java:745)
1 error
at
org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:309)
at
org.codehaus.groovy.control.ErrorCollector.addException(ErrorCollector.java:155)
at
org.codehaus.groovy.control.SourceUnit.addException(SourceUnit.java:345)
at
groovy.grape.GrabAnnotationTransformation.visit(GrabAnnotationTransformation.java:367)
at
org.codehaus.groovy.transform.ASTTransformationVisitor$3.call(ASTTransformationVisitor.java:319)
at
org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:928)
at
org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:590)
at
org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:566)
at
org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:543)
at
groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:297)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:267)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:692)
at groovy.lang.GroovyShell.parse(GroovyShell.java:704)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:588)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:627)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:598)
at
GrapeAndGroovyShellConcurrencyTest$1.run(GrapeAndGroovyShellConcurrencyTest.java:64)
at java.lang.Thread.run(Thread.java:745)
------------
Stacktrace 2
------------
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup
failed:
file:/var/folders/38/r0n49vmn7zg5dffk79_tgpl80000gn/T/junit124846036508580912/Util7.groovy:
1: unable to resolve class com.google.common.base.Ascii
@ line 1, column 1.
@Grab('com.google.guava:guava:18.0')
^
1 error
at
org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:309)
at
org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:943)
at
org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:590)
at
org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:539)
at
groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:297)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:267)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:692)
at groovy.lang.GroovyShell.parse(GroovyShell.java:704)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:588)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:627)
at groovy.lang.GroovyShell.evaluate(GroovyShell.java:598)
at
GrapeAndGroovyShellConcurrencyTest$1.run(GrapeAndGroovyShellConcurrencyTest.java:64)
at java.lang.Thread.run(Thread.java:745)
