Hi Mihai, from a quick look you indeed seem to have hit a bug, filing a Jira ticket would be great.
In principle all the information you need around filing a ticket are available here: https://calcite.apache.org/develop/#contributing If there is something missing there, please let us know here and we might also consider improving the doc. If you don't plan to tackle the issue, the repro instructions you gave seem enough to me, if you file a PR please add a new test covering exactly the error you are fixing (do not modify the existing one, unless you can make it cover both the original and the new case elegantly) so that we avoid regressions over time. If you want to assign the ticket to yourself, you can request the "contributor" role for Jira in this list (I don't have permissions for granting that, it's generally Francis who takes care of that and he is very responsive). Best regards, Alessandro On Sat, 17 Sept 2022 at 02:54, Mihai Budiu <[email protected]> wrote: > Hello, > > I think I have hit a bug in the MultiJoinOptimizeBushyRule planner rule, > but I could not figure out how to file an issue on the JIRA. > If you give me some guidance I can file the issue there directly. > > The reproduction is easy: just modify the following test case from > PlannerTest.java: > > --- a/core/src/test/java/org/apache/calcite/tools/PlannerTest.java > +++ b/core/src/test/java/org/apache/calcite/tools/PlannerTest.java > @@ -1005,7 +1005,7 @@ private void checkJoinNWay(int n) throws Exception { > private void checkHeuristic(String sql, String expected) throws > Exception { > Planner planner = getPlanner(null, > - Programs.heuristicJoinOrder(Programs.RULE_SET, false, 0)); > + Programs.heuristicJoinOrder(Programs.RULE_SET, true, 0)); > SqlNode parse = planner.parse(sql); > SqlNode validate = planner.validate(parse); > RelNode convert = planner.rel(validate).rel; > > Then the test fails with the exception shown below. This happens with the > latest version of calcite, the main branch. > It looks like the rule does not account for the fact that outer joins can > produce results with a different nullability than the input relations. > > Thank you, > Mihai > > java.lang.RuntimeException: Error while applying rule > MultiJoinOptimizeBushyRule, args > [rel#44:MultiJoin.NONE.[](input#0=RelSubset#42,input#1=RelSubset#43,joinFilter=true,isFullOuterJoin=false,joinTypes=[RIGHT, > INNER],outerJoinConditions=[=($0, $10), NULL],projFields=[ALL, ALL])] > > at > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:250) > at > org.apache.calcite.plan.volcano.IterativeRuleDriver.drive(IterativeRuleDriver.java:59) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:523) > at > org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:318) > at > org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:337) > at > org.apache.calcite.tools.Programs.lambda$heuristicJoinOrder$1(Programs.java:223) > at > org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:373) > at > org.apache.calcite.tools.PlannerTest.checkHeuristic(PlannerTest.java:1014) > at > org.apache.calcite.tools.PlannerTest.testHeuristicRightJoin(PlannerTest.java:1003) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725) > at > org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) > at > org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) > at > org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149) > at > org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140) > at > org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84) > at > org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115) > at > org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105) > at > org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) > at > org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) > at > org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) > at > org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) > at > org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104) > at > org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98) > at > org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214) > at > org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) > at > org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210) > at > org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135) > at > org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151) > at > org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) > at > org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) > at > org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) > at java.util.ArrayList.forEach(ArrayList.java:1257) > at > org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) > at > org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) > at > org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) > at > org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) > at java.util.ArrayList.forEach(ArrayList.java:1257) > at > org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155) > at > org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141) > at > org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139) > at > org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138) > at > org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95) > at > org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35) > at > org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57) > at > org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67) > at > org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52) > at > org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114) > at > org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86) > at > org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86) > at > org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53) > at > com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71) > at > com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38) > at > com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11) > at > com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35) > at > com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235) > at > com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54) > Caused by: java.lang.RuntimeException: Error occurred while applying rule > MultiJoinOptimizeBushyRule > at > org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:157) > at > org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:269) > at > org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:284) > at > org.apache.calcite.rel.rules.MultiJoinOptimizeBushyRule.onMatch(MultiJoinOptimizeBushyRule.java:292) > at > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:223) > ... 77 more > Caused by: java.lang.IllegalArgumentException: Type mismatch: > rel rowtype: RecordType(JavaType(int) NOT NULL empid, JavaType(int) NOT > NULL deptno, JavaType(class java.lang.String) name, JavaType(float) NOT > NULL salary, JavaType(class java.lang.Integer) commission, JavaType(int) > NOT NULL deptno0, JavaType(class java.lang.String) name0, > RecordType(JavaType(int) NOT NULL empid, JavaType(int) NOT NULL deptno, > JavaType(class java.lang.String) name, JavaType(float) NOT NULL salary, > JavaType(class java.lang.Integer) commission) NOT NULL ARRAY NOT NULL > employees, JavaType(int) NOT NULL x, JavaType(int) NOT NULL y, > JavaType(int) NOT NULL empid0, JavaType(class java.lang.String) name1) NOT > NULL > equiv rowtype: RecordType(JavaType(class java.lang.Integer) empid, > JavaType(class java.lang.Integer) deptno, JavaType(class java.lang.String) > name, JavaType(class java.lang.Float) salary, JavaType(class > java.lang.Integer) commission, JavaType(class java.lang.Integer) deptno0, > JavaType(class java.lang.String) name0, RecordType(JavaType(int) NOT NULL > empid, JavaType(int) NOT NULL deptno, JavaType(class java.lang.String) > name, JavaType(float) NOT NULL salary, JavaType(class java.lang.Integer) > commission) NOT NULL ARRAY employees, JavaType(class java.lang.Integer) x, > JavaType(class java.lang.Integer) y, JavaType(int) NOT NULL empid0, > JavaType(class java.lang.String) name1) NOT NULL > Difference: > empid: JavaType(int) NOT NULL -> JavaType(class java.lang.Integer) > deptno: JavaType(int) NOT NULL -> JavaType(class java.lang.Integer) > salary: JavaType(float) NOT NULL -> JavaType(class java.lang.Float) > deptno0: JavaType(int) NOT NULL -> JavaType(class java.lang.Integer) > employees: RecordType(JavaType(int) NOT NULL empid, JavaType(int) NOT NULL > deptno, JavaType(class java.lang.String) name, JavaType(float) NOT NULL > salary, JavaType(class java.lang.Integer) commission) NOT NULL ARRAY NOT > NULL -> RecordType(JavaType(int) NOT NULL empid, JavaType(int) NOT NULL > deptno, JavaType(class java.lang.String) name, JavaType(float) NOT NULL > salary, JavaType(class java.lang.Integer) commission) NOT NULL ARRAY > x: JavaType(int) NOT NULL -> JavaType(class java.lang.Integer) > y: JavaType(int) NOT NULL -> JavaType(class java.lang.Integer) > > at > org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:592) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:613) > at > org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:144) > ... 81 more >
