Hi,
Now we come to the problem similar as the one at
https://github.com/apache/calcite/pull/2433#issuecomment-860024076,
a simple sql "select max(id) from s.t1" caused the following exception, users
report upgrading janino to 3.1.6 can fix this problem, but as we discussed
before janino 3.1.6 is not fully compatible with calcite 1.29.0 because of the
dependency issue.
at org.apache.calcite.avatica.Helper.wrap(Helper.java:37)
~[avatica-core-1.20.0.jar:1.20.0]
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:130)
~[calcite-core-1.29.0.jar:1.29.0]
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1130)
~[calcite-core-1.29.0.jar:1.29.0]
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepare_(CalcitePrepareImpl.java:1032)
~[calcite-core-1.29.0.jar:1.29.0]
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.prepareRel(CalcitePrepareImpl.java:988)
~[calcite-core-1.29.0.jar:1.29.0]
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:668)
~[calcite-core-1.29.0.jar:1.29.0]
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:513)
~[calcite-core-1.29.0.jar:1.29.0]
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:483)
~[calcite-core-1.29.0.jar:1.29.0]
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:249)
~[calcite-core-1.29.0.jar:1.29.0]
at
org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:229)
~[calcite-core-1.29.0.jar:1.29.0]
... 13 common frames omitted
Caused by: org.codehaus.commons.compiler.InternalCompilerException: Compiling
"Baz" in File '/Users/linda/Downloads/janino/janino6606844895950068558.java',
Line 1, Column 1: File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 41, Column
45: Compiling "bind(org.apache.calcite.DataContext root)": File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 60, Column
64
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:369)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.access$000(UnitCompiler.java:231)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler.java:333)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler.java:330)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.Java$CompilationUnit.accept(Java.java:367)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:330)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:245)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:294)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:288)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.ClassBodyEvaluator.cook(ClassBodyEvaluator.java:267)
~[janino-3.1.4.jar:na]
at org.codehaus.commons.compiler.Cookable.cook(Cookable.java:52)
~[commons-compiler-3.1.4.jar:na]
at
org.codehaus.janino.ClassBodyEvaluator.createInstance(ClassBodyEvaluator.java:428)
~[janino-3.1.4.jar:na]
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.getBindable(EnumerableInterpretable.java:166)
~[calcite-core-1.29.0.jar:1.29.0]
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:127)
~[calcite-core-1.29.0.jar:1.29.0]
... 21 common frames omitted
Caused by: org.codehaus.commons.compiler.InternalCompilerException: File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 41, Column
45: Compiling "bind(org.apache.calcite.DataContext root)": File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 60, Column
64
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3222)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1379)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1352)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:800)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:412)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:231)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:391)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:386)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1692)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:386)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:359)
~[janino-3.1.4.jar:na]
... 34 common frames omitted
Caused by: java.lang.RuntimeException: File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 60, Column
64
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2639)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.access$2700(UnitCompiler.java:231)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$6.visitLocalVariableDeclarationStatement(UnitCompiler.java:1539)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$6.visitLocalVariableDeclarationStatement(UnitCompiler.java:1523)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.Java$LocalVariableDeclarationStatement.accept(Java.java:3840)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1523)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1607)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3531)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3218)
~[janino-3.1.4.jar:na]
... 44 common frames omitted
Caused by: org.codehaus.commons.compiler.InternalCompilerException: File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 60, Column
80: Compiling "new
org.apache.calcite.adapter.enumerable.BasicAggregateLambdaFactory(new
org.apache.calcite.linq4j.function.Function0() { ... }, accumulatorAdders)":
File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 61,
Column 5: Compiling "new org.apache.calcite.linq4j.function.Function0() { ...
}": File '/Users/linda/Downloads/janino/janino6606844895950068558.java', Line
62, Column 21: Compiling "apply()": File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 70, Column
17: org.codehaus.commons.compiler.InternalCompilerException: Operand stack
underflow
at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5731)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2622)
~[janino-3.1.4.jar:na]
... 52 common frames omitted
Caused by: org.codehaus.commons.compiler.InternalCompilerException: File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 61, Column
5: Compiling "new org.apache.calcite.linq4j.function.Function0() { ... }": File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 62, Column
21: Compiling "apply()": File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 70, Column
17: org.codehaus.commons.compiler.InternalCompilerException: Operand stack
underflow
at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5731)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler.invokeConstructor(UnitCompiler.java:8297)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5468)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.access$9700(UnitCompiler.java:231)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$15.visitNewClassInstance(UnitCompiler.java:4643)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$15.visitNewClassInstance(UnitCompiler.java:4604)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.Java$NewClassInstance.accept(Java.java:5560)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4604)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5729)
~[janino-3.1.4.jar:na]
... 53 common frames omitted
Caused by: org.codehaus.commons.compiler.InternalCompilerException: File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 62, Column
21: Compiling "apply()": File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 70, Column
17: org.codehaus.commons.compiler.InternalCompilerException: Operand stack
underflow
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3222)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1379)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler.compileDeclaredMethods(UnitCompiler.java:1352)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:800)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:977)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:947)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.access$200(UnitCompiler.java:231)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$2.visitAnonymousClassDeclaration(UnitCompiler.java:389)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$2.visitAnonymousClassDeclaration(UnitCompiler.java:386)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.Java$AnonymousClassDeclaration.accept(Java.java:1375)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:386)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:5571)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.access$9400(UnitCompiler.java:231)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$15.visitNewAnonymousClassInstance(UnitCompiler.java:4640)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$15.visitNewAnonymousClassInstance(UnitCompiler.java:4604)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.Java$NewAnonymousClassInstance.accept(Java.java:5608)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:4604)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:5729)
~[janino-3.1.4.jar:na]
... 61 common frames omitted
Caused by: org.codehaus.commons.compiler.InternalCompilerException: File
'/Users/linda/Downloads/janino/janino6606844895950068558.java', Line 70, Column
17: org.codehaus.commons.compiler.InternalCompilerException: Operand stack
underflow
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2393)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.access$1800(UnitCompiler.java:231)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1530)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$6.visitExpressionStatement(UnitCompiler.java:1523)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.Java$ExpressionStatement.accept(Java.java:3198)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:1523)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler.compileStatements(UnitCompiler.java:1607)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3531)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3218)
~[janino-3.1.4.jar:na]
... 78 common frames omitted
Caused by: org.codehaus.commons.compiler.InternalCompilerException: Operand
stack underflow
at org.codehaus.janino.StackMap.peekOperand(StackMap.java:94)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.CodeContext.popOperand(CodeContext.java:1349)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.putfield(UnitCompiler.java:12363)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:6498)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.access$13400(UnitCompiler.java:231)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$20.visitFieldAccess(UnitCompiler.java:6480)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$20.visitFieldAccess(UnitCompiler.java:6475)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.Java$FieldAccess.accept(Java.java:4668)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:6475)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compileSet2(UnitCompiler.java:6489)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.access$13200(UnitCompiler.java:231)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$20.visitAmbiguousName(UnitCompiler.java:6478)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$20.visitAmbiguousName(UnitCompiler.java:6475)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.Java$AmbiguousName.accept(Java.java:4574)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compileSet(UnitCompiler.java:6475)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:3992)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.access$6000(UnitCompiler.java:231)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$12.visitAssignment(UnitCompiler.java:3944)
~[janino-3.1.4.jar:na]
at
org.codehaus.janino.UnitCompiler$12.visitAssignment(UnitCompiler.java:3924)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.Java$Assignment.accept(Java.java:4835)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:3924)
~[janino-3.1.4.jar:na]
at org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:2391)
~[janino-3.1.4.jar:na]
... 86 common frames omitted
With janino 3.1.4
------------------ Original ------------------
From: "xiaobo ";<[email protected]>;
Send time: Saturday, Feb 12, 2022 9:14 PM
To: "dev"<[email protected]>;
Subject: Re: calcite multi-threading problem
3.1.4 of janino and commons-compiler works now.
------------------ Original ------------------
From: "xiaobo ";<[email protected]>;
Send time: Saturday, Feb 12, 2022 8:04 PM
To: "dev"<[email protected]>;
Subject: Re: calcite multi-threading problem
but strange enough, the same code works in a single thread.
------------------ Original ------------------
From: "xiaobo ";<[email protected]>;
Send time: Saturday, Feb 12, 2022 8:02 PM
To: "dev"<[email protected]>;
Subject: Re: calcite multi-threading problem
we checked the janino release note page
http://janino-compiler.github.io/janino/changelog.html
Fixed issue #141: Unable to find org.codehaus.commons.compiler.properties in
java 11: ICompilerFactories were loaded through the current thread's "context
class loader", which is a bad choice in some environments. Thus, the methods
"getDefaultCompilerFactory()", "getAllCompilerFactories()" and
"getCompilerFactory(className)" were duplicated and augmented with a
"classLoader" parameter.
what does this mean? the getDefaultCompilerFactory method with a classloader
parameter was removed intentionally?
but it seems calcite still needs it , so what version of janino should we use
for the latest calcite-core?
------------------ Original ------------------
From: "xiaobo ";<[email protected]>;
Send time: Saturday, Feb 12, 2022 12:01 PM
To: "dev"<[email protected]>;
Subject: Re: calcite multi-threading problem
In our use case, the Java class which opens calcite connection is contained in
a external Java jar file, which is loaded dynamicly through classloaders other
than the default systemloader like this:
try {
ClassLoader loader = URLClassLoader.newInstance(
new URL[] { new URL("jar:file:" + path + "!/") },
getClass().getClassLoader());
Class<?> clazz = Class.forName(classPath, true, loader);
we mention this because the error message shows a classloader parameter.
------------------ Original ------------------
From: "xiaobo ";<[email protected]>;
Send time: Saturday, Feb 12, 2022 11:56 AM
To: "dev"<[email protected]>;
Subject: Re: calcite multi-threading problem
And if we remove calcite from our project, janino is not in "resolved
depencies" library list of pom.xml,
So we think it not the janino conflic problem.
------------------ Original ------------------
From: "xiaobo ";<[email protected]>;
Send time: Saturday, Feb 12, 2022 9:49 AM
To: "dev"<[email protected]>;
Subject: Re: calcite multi-threading problem
can you share the solution for the problem, we are using the following
libraries, and it seems janino 3.1.6 is only dependent by calcite-core.
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.encoding>UTF-8</maven.compiler.encoding>
<java.version>9</java.version>
<maven.compiler.source>9</maven.compiler.source>
<maven.compiler.target>9</maven.compiler.target>
<log4j2.version>2.17.1</log4j2.version>
<drools.version>7.64.0.Final</drools.version>
</properties>
<groupId>org.apache.calcite</groupId>
<artifactId>calcite-core</artifactId>
<version>1.29.0</version>
</dependency>
<dependency>
<groupId>org.apache.calcite.avatica</groupId>
<artifactId>avatica-core</artifactId>
<version>1.20.0</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-core</artifactId>
<version>${drools.version}</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-compiler</artifactId>
<version>${drools.version}</version>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-decisiontables</artifactId>
<version>${drools.version}</version>
</dependency>
------------------ Original ------------------
From: "xiong duan";<[email protected]>;
Send time: Saturday, Feb 12, 2022 8:43 AM
To: "dev"<[email protected]>;
Subject: Re: calcite multi-threading problem
Yes, It is a dependency issue. I have met the same problem. +1 for Dmitry.
Dmitry Sysolyatin <[email protected]> 于2022年2月11日周五 21:18写道：
> Actually, I had the same problem with spark. Spark 3.2.1 uses the old
> version of janino, calcite 1.29 uses the new version of janino and they are
> not compatible.
> If I downgrade janino version to version which spark uses I got an error
> not method found exception
> If I upgrade the janino version to the version which calcite uses I get the
> same error but with another method.
>
> As a result, I built a separate calcite jar package where I shaded janino
> dependency
>
>
>
> On Fri, Feb 11, 2022 at 2:39 PM stanilovsky evgeny <
> [email protected]> wrote:
>
> > for example on a current calcite branch:
> > grep janino gradle.properties
> > janino.version=3.1.6
> >
> >
> > > yes, we use drools in the same project, and drools uses janino
> > > too, is there a version list of janino which calcite supports?
> > >
> > >
> > >
> > >
> > > ---Original---
> > > From: "Dmitry Sysolyatin"<[email protected]>
> > > Date: Fri, Feb 11, 2022 18:30 PM
> > > To: "dev"<[email protected]>;
> > > Subject: Re: calcite multi-threading problem
> > >
> > >
> > > Hi!
> > >
> >
> It looks like a dependency issue, not like a multi-threading issue. Most
> > >
> >
> likely you have some dependency that use another version of janino but
> > > calcite requires another one
> > >
> > >
> >
> On Fri, Feb 11, 2022 at 12:18 PM xiaobo <[email protected]
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > >
> > >
> >
> > we open a calcite connection with ReflectiveSchema in one thread, and
> > >
> >
> > execute sqls in other threads against the connection, but failed with
> > > > this error
> > > >
> > > >
> > > >
> > > >
> > > > com.google.common.util.concurrent.ExecutionError:
> > > > java.lang.NoSuchMethodError:
> > > >
> > >
> >
> > org.codehaus.commons.compiler.CompilerFactoryFactory.getDefaultCompilerFactory(Ljava/lang/ClassLoader;)
> > > >
> > > > Lorg/codehaus/commons/compiler/ICompilerFactory;
> > > >
> > > >
> > >
> >
> > do we miss anything regarding to multi-threading with calcite?
> >
>
