Back to the original problem (NPE at AggregateNode.getAccumulator, line 150), I've hit this bug again and I'm still struggling to reproduce it through a test case or evade it altogether.
The bug is reproducible through my custom adaptor using this query: SELECT cn."COUNTRY_NAME", count(prd."PROD_NAME") Product_Count, count( *DISTINCT* (prd."PROD_NAME")) Product_Distinct FROM SALES.SALES1 s LEFT OUTER JOIN SALES.PRODUCTS prd ON s."PROD_ID" = prd."PROD_ID" LEFT OUTER JOIN SALES.CUSTOMERS c ON s."CUSTOMER_ID" = c."CUST_ID" LEFT OUTER JOIN SALES.COUNTRIES cn ON c."COUNTRY_ID" = cn."COUNTRY_ID" GROUP BY cn."COUNTRY_NAME" I noticed that if I remove the DISTINCT keyword (highlighted in bold), the bug isn't activated. I developed my adaptor by following the Druid adapter way, I think I missed something but I can't decide what it is. I've been thinking that since this piece of code should not be activated, as it will always throw an exception. Could someone please tell me why would an executing query hit in the first place ? Or how should I configure my adaptor, planner, query..etc to NOT hit that code ? Thanks, Gelbana On Wed, Apr 25, 2018 at 9:56 PM, Muhammad Gelbana <[email protected]> wrote: > Thank you! That helped a lot. > > Thanks, > Gelbana > > On Wed, Apr 25, 2018 at 12:00 PM, Stamatis Zampetakis <[email protected]> > wrote: > >> Hi Gelbana, >> >> I haven't looked at your snippet but if you want to debug into generated >> code have a look at the following link: >> https://calcite.apache.org/docs/howto.html#debugging-generat >> ed-classes-in-intellij >> >> Best, >> Stamatis >> >> 2018-04-21 14:07 GMT+02:00 Muhammad Gelbana <[email protected]>: >> >> > I'm trying to create the test case for this issue but I can't run a >> simple >> > query as a start. >> > >> > I'm facing the following exception >> > >> > java.sql.SQLException: exception while executing query: null >> > >> > at org.apache.calcite.avatica.Helper.createException(Helper.jav >> a:56) >> > at org.apache.calcite.avatica.Helper.createException(Helper.jav >> a:41) >> > at >> > org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal( >> > AvaticaConnection.java:540) >> > at >> > org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery( >> > AvaticaPreparedStatement.java:133) >> > at >> > org.apache.calcite.tools.FrameworksTest.aggNPE(FrameworksTest.java:184) >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >> ssorImpl.java: >> > 62) >> > at >> > sun.reflect.DelegatingMethodAccessorImpl.invoke( >> > DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:498) >> > at >> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( >> > FrameworkMethod.java:50) >> > at >> > org.junit.internal.runners.model.ReflectiveCallable.run( >> > ReflectiveCallable.java:12) >> > at >> > org.junit.runners.model.FrameworkMethod.invokeExplosively( >> > FrameworkMethod.java:47) >> > at >> > org.junit.internal.runners.statements.InvokeMethod. >> > evaluate(InvokeMethod.java:17) >> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) >> > at >> > org.junit.runners.BlockJUnit4ClassRunner.runChild( >> > BlockJUnit4ClassRunner.java:78) >> > at >> > org.junit.runners.BlockJUnit4ClassRunner.runChild( >> > BlockJUnit4ClassRunner.java:57) >> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) >> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) >> > at org.junit.runners.ParentRunner.runChildren(ParentRunner. >> java:288) >> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) >> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) >> > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) >> > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) >> > at >> > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs( >> > JUnit4IdeaTestRunner.java:68) >> > at >> > com.intellij.rt.execution.junit.IdeaTestRunner$Repeater. >> > startRunnerWithArgs(IdeaTestRunner.java:47) >> > at >> > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart( >> > JUnitStarter.java:242) >> > at >> > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) >> > >> > *Caused by: java.lang.NullPointerException at Baz.bind(Unknown >> Source)* >> > at >> > org.apache.calcite.adapter.enumerable.EnumerableBindable. >> > bind(EnumerableBindable.java:70) >> > at >> > org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable( >> > CalcitePrepare.java:365) >> > at >> > org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable( >> > CalciteConnectionImpl.java:301) >> > at >> > org.apache.calcite.jdbc.CalciteMetaImpl._createIterable( >> > CalciteMetaImpl.java:559) >> > at >> > org.apache.calcite.jdbc.CalciteMetaImpl.createIterable( >> > CalciteMetaImpl.java:550) >> > at >> > org.apache.calcite.avatica.AvaticaResultSet.execute( >> > AvaticaResultSet.java:172) >> > at >> > org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResu >> ltSet.java:67) >> > at >> > org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResu >> ltSet.java:44) >> > at >> > org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal( >> > AvaticaConnection.java:536) >> > ... 24 more >> > >> > This is the test case I'm working on >> > >> > @Test >> > public void aggNPE() throws SqlParseException, ValidationException, >> > RelConversionException, SQLException { >> > Function<Holder, Object> hook = (o1) -> { >> > o1.set(true); >> > return null; >> > }; >> > Hook.ENABLE_BINDABLE.add(hook); >> > >> > SchemaPlus rootSchema = Frameworks.createRootSchema(true); >> > final FrameworkConfig config = Frameworks.newConfigBuilder() >> > .defaultSchema(CalciteAssert.addSchema(rootSchema, >> > CalciteAssert.SchemaSpec.JDBC_FOODMART)) >> > .ruleSets(RuleSets.ofList(Bindables.RULES)) >> > .build(); >> > >> > Planner planner = Frameworks.getPlanner(config); >> > SqlNode parsed = planner.parse("SELECT * FROM \"sales_fact_1997\""); >> > SqlNode validated = planner.validate(parsed); >> > RelRoot rel = planner.rel(validated); >> > >> > PreparedStatement statement = RelRunners.run(rel.rel); >> > ResultSet rs = statement.executeQuery(); >> > >> > rs.close(); >> > statement.close(); >> > } >> > >> > I'm trying to be as close as possible to what I have in my schema so I >> > enabled the bindable convention and enabled the bindables rules. What >> am I >> > doing wrong here to cause this NPE ? >> > I can't debug into the *Baz* class to figure out what's going on as the >> > class seems to be generated at runtime or something. >> > >> > >> > Thanks, >> > Gelbana >> > >> > On Thu, Mar 29, 2018 at 1:34 AM, Muhammad Gelbana <[email protected]> >> > wrote: >> > >> > > With pleasure. I'll do that ASAP. >> > > >> > > Thanks, >> > > Gelbana >> > > >> > > On Thu, Mar 29, 2018 at 12:38 AM, Shuyi Chen <[email protected]> >> wrote: >> > > >> > >> It will be great if you can provide a test to reproduce the problem, >> and >> > >> attach it in the JIRA. >> > >> >> > >> On Wed, Mar 28, 2018 at 2:50 PM, Muhammad Gelbana < >> [email protected]> >> > >> wrote: >> > >> >> > >> > Absolutely: https://issues.apache.org/jira/browse/CALCITE-2228 >> > >> > >> > >> > Please tell me if I should modify any of the information I >> supplied. >> > >> Like >> > >> > the affected version\component. >> > >> > >> > >> > Thanks, >> > >> > Gelbana >> > >> > >> > >> > On Wed, Mar 28, 2018 at 11:26 PM, Julian Hyde <[email protected]> >> > wrote: >> > >> > >> > >> > > It does indeed look as if that piece of code (AggregateNode.java >> > line >> > >> > 149) >> > >> > > has never been run. Can you please log a JIRA case? >> > >> > > >> > >> > > > On Mar 28, 2018, at 1:42 PM, Muhammad Gelbana < >> > [email protected]> >> > >> > > wrote: >> > >> > > > >> > >> > > > Looking at this line >> > >> > > > <https://github.com/apache/calcite/blob/master/core/src/ >> > >> > > main/java/org/apache/calcite/interpreter/AggregateNode.java# >> L150>, >> > >> > > > and the referenced instance variable >> > >> > > > <https://github.com/apache/calcite/blob/master/core/src/ >> > >> > > main/java/org/apache/calcite/adapter/enumerable/AggImpState. >> > >> java#L33>, >> > >> > > > it looks to me that *int stateSize = agg.state.size* will >> always >> > >> throw >> > >> > a >> > >> > > > NPE because the *agg.state* object reference is never >> initialized >> > >> > before >> > >> > > > it's being used ! >> > >> > > > >> > >> > > > I'm facing the following exception >> > >> > > > >> > >> > > > java.sql.SQLException: Error while executing SQL "SELECT >> > >> > MAX(COUNTRY_ID), >> > >> > > > MAX(COUNTRY_TOTAL_ID) FROM SALES.COUNTRIES": null >> > >> > > > at org.apache.calcite.avatica.Hel >> per.createException(Helper. >> > >> > java:56) >> > >> > > > at org.apache.calcite.avatica.Hel >> per.createException(Helper. >> > >> > java:41) >> > >> > > > at >> > >> > > > org.apache.calcite.avatica.AvaticaStatement.executeInternal( >> > >> > > AvaticaStatement.java:156) >> > >> > > > at >> > >> > > > org.apache.calcite.avatica.AvaticaStatement.executeQuery( >> > >> > > AvaticaStatement.java:218) >> > >> > > > at >> > >> > > > com.mg.sqlinterface.data.provider.calcite.internal. >> > >> > > Test_SQLi.testMG(Test_SQLi.java:35) >> > >> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> > >> > > > at >> > >> > > > sun.reflect.NativeMethodAccessorImpl.invoke( >> > >> > > NativeMethodAccessorImpl.java:62) >> > >> > > > at >> > >> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke( >> > >> > > DelegatingMethodAccessorImpl.java:43) >> > >> > > > at java.lang.reflect.Method.invoke(Method.java:498) >> > >> > > > at >> > >> > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( >> > >> > > FrameworkMethod.java:50) >> > >> > > > at >> > >> > > > org.junit.internal.runners.model.ReflectiveCallable.run( >> > >> > > ReflectiveCallable.java:12) >> > >> > > > at >> > >> > > > org.junit.runners.model.FrameworkMethod.invokeExplosively( >> > >> > > FrameworkMethod.java:47) >> > >> > > > at >> > >> > > > org.junit.internal.runners.statements.InvokeMethod. >> > >> > > evaluate(InvokeMethod.java:17) >> > >> > > > at org.junit.runners.ParentRunner.runLeaf( >> > ParentRunner.java:325) >> > >> > > > at >> > >> > > > org.junit.runners.BlockJUnit4ClassRunner.runChild( >> > >> > > BlockJUnit4ClassRunner.java:78) >> > >> > > > at >> > >> > > > org.junit.runners.BlockJUnit4ClassRunner.runChild( >> > >> > > BlockJUnit4ClassRunner.java:57) >> > >> > > > at org.junit.runners.ParentRunner >> $3.run(ParentRunner.java:290) >> > >> > > > at org.junit.runners.ParentRunner$1.schedule( >> > ParentRunner.java: >> > >> 71) >> > >> > > > at org.junit.runners.ParentRunner.runChildren( >> > >> > ParentRunner.java:288) >> > >> > > > at org.junit.runners.ParentRunner.access$000( >> > ParentRunner.java: >> > >> 58) >> > >> > > > at org.junit.runners.ParentRunner$2.evaluate( >> > ParentRunner.java: >> > >> 268) >> > >> > > > at >> > >> > > > org.junit.internal.runners.statements.RunBefores. >> > >> > > evaluate(RunBefores.java:26) >> > >> > > > at >> > >> > > > org.junit.internal.runners.statements.RunAfters.evaluate( >> > >> > > RunAfters.java:27) >> > >> > > > at org.junit.runners.ParentRunner >> .run(ParentRunner.java:363) >> > >> > > > at >> > >> > > > org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference. >> run( >> > >> > > JUnit4TestReference.java:86) >> > >> > > > at >> > >> > > > org.eclipse.jdt.internal.junit.runner.TestExecution. >> > >> > > run(TestExecution.java:38) >> > >> > > > at >> > >> > > > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner. >> > >> > > runTests(RemoteTestRunner.java:459) >> > >> > > > at >> > >> > > > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner. >> > >> > > runTests(RemoteTestRunner.java:678) >> > >> > > > at >> > >> > > > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner. >> > >> > > run(RemoteTestRunner.java:382) >> > >> > > > at >> > >> > > > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner. >> > >> > > main(RemoteTestRunner.java:192) >> > >> > > > Caused by: *java.lang.NullPointerException* >> > >> > > > at org.apache.calcite.interpreter.AggregateNode. >> > getAccumulator( >> > >> > > > *AggregateNode.java:150*) >> > >> > > > at >> > >> > > > org.apache.calcite.interpreter.AggregateNode.< >> > >> > > init>(AggregateNode.java:86) >> > >> > > > at >> > >> > > > org.apache.calcite.interpreter.Nodes$CoreCompiler.visit( >> > >> Nodes.java:47) >> > >> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> > >> > > > at >> > >> > > > sun.reflect.NativeMethodAccessorImpl.invoke( >> > >> > > NativeMethodAccessorImpl.java:62) >> > >> > > > at >> > >> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke( >> > >> > > DelegatingMethodAccessorImpl.java:43) >> > >> > > > at java.lang.reflect.Method.invoke(Method.java:498) >> > >> > > > at >> > >> > > > org.apache.calcite.util.ReflectUtil.invokeVisitorInternal( >> > >> > > ReflectUtil.java:257) >> > >> > > > at >> > >> > > > org.apache.calcite.util.ReflectUtil.invokeVisitor( >> > >> > ReflectUtil.java:214) >> > >> > > > at >> > >> > > > org.apache.calcite.util.ReflectUtil$1.invokeVisitor( >> > >> > > ReflectUtil.java:464) >> > >> > > > at >> > >> > > > org.apache.calcite.interpreter.Interpreter$Compiler.visit( >> > >> Interpreter. >> > >> > > java:480) >> > >> > > > at >> > >> > > > org.apache.calcite.interpreter.Interpreter$Compiler.visitRoot( >> > >> > > Interpreter.java:437) >> > >> > > > at >> > >> > > > org.apache.calcite.interpreter.Interpreter.<init>( >> > >> Interpreter.java:78) >> > >> > > > at Baz.bind(Unknown Source) >> > >> > > > at >> > >> > > > org.apache.calcite.jdbc.CalcitePrepare$ >> > CalciteSignature.enumerable( >> > >> > > CalcitePrepare.java:365) >> > >> > > > at >> > >> > > > org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable( >> > >> > > CalciteConnectionImpl.java:301) >> > >> > > > at >> > >> > > > org.apache.calcite.jdbc.CalciteMetaImpl._createIterable( >> > >> > > CalciteMetaImpl.java:559) >> > >> > > > at >> > >> > > > org.apache.calcite.jdbc.CalciteMetaImpl.createIterable( >> > >> > > CalciteMetaImpl.java:550) >> > >> > > > at >> > >> > > > org.apache.calcite.avatica.AvaticaResultSet.execute( >> > >> > > AvaticaResultSet.java:204) >> > >> > > > at >> > >> > > > org.apache.calcite.jdbc.CalciteResultSet.execute( >> > >> > > CalciteResultSet.java:67) >> > >> > > > at >> > >> > > > org.apache.calcite.jdbc.CalciteResultSet.execute( >> > >> > > CalciteResultSet.java:44) >> > >> > > > at >> > >> > > > org.apache.calcite.avatica.AvaticaConnection$1.execute( >> > >> > > AvaticaConnection.java:630) >> > >> > > > at >> > >> > > > org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute( >> > >> > > CalciteMetaImpl.java:619) >> > >> > > > at >> > >> > > > org.apache.calcite.avatica.AvaticaConnection. >> > >> > prepareAndExecuteInternal( >> > >> > > AvaticaConnection.java:638) >> > >> > > > at >> > >> > > > org.apache.calcite.avatica.AvaticaStatement.executeInternal( >> > >> > > AvaticaStatement.java:149) >> > >> > > > ... 27 more >> > >> > > > >> > >> > > > >> > >> > > > Thanks, >> > >> > > > Gelbana >> > >> > > > >> > >> > > >> > >> > > >> > >> > >> > >> >> > >> >> > >> >> > >> -- >> > >> "So you have to trust that the dots will somehow connect in your >> > future." >> > >> >> > > >> > > >> > >> > >
