[
https://issues.apache.org/jira/browse/CALCITE-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17211440#comment-17211440
]
Julian Hyde edited comment on CALCITE-4317 at 10/10/20, 12:28 AM:
------------------------------------------------------------------
[~julianhyde], we did see the above issue both in LoptOptimizeJoinRule and
ProjectJoinTransposeRule but i was unable to repro the above issue in
LoptOptimizeJoinRule and after some research, i found out that this issue has
been fixed already( we need to cherry-pick this change). See CALCITE-2720.
Since we have already fixed the issue of empty aggregate in
LoptOptimizeJoinRule, i think we should also fix the same for
ProjectJoinTransposeRule. I could not create the query which would fail for
ProjectJoinTransposeRule since we do a lot of transformations, but this simple
test will repro the issue:
{code:java}
@Test void testProjectJoinTransposeRuleOnAggWithNoFields() {
final RelBuilder relBuilder =
RelBuilder.create(RelBuilderTest.config().build());
// Build a rel equivalent to sql:
// SELECT name FROM (SELECT count(*) cnt_star, count(empno) cnt_en FROM
sales.emp)
// cross join sales.dept
// limit 10
RelNode left = relBuilder.scan("DEPT").build();
RelNode right = relBuilder.scan("EMP")
.project(ImmutableList.of(relBuilder.getRexBuilder().makeExactLiteral(BigDecimal.ZERO)),
ImmutableList.of("DUMMY"))
.aggregate(relBuilder.groupKey()).build();
RelNode plan = relBuilder.push(left)
.push(right)
.join(JoinRelType.INNER,
relBuilder.getRexBuilder().makeLiteral(true))
.project(relBuilder.field("DEPTNO"))
.build();
HepProgram program = new HepProgramBuilder()
.addRuleInstance(CoreRules.PROJECT_JOIN_TRANSPOSE)
.build();
HepPlanner hepPlanner = new HepPlanner(program);
hepPlanner.setRoot(plan);
hepPlanner.findBestExp();
}
{code}
Regardless, we should fix the RelFieldTrimmer as well, as we discussed above.
was (Author: rafay):
[~julianhyde], we did see the above issue both in LoptOptimizeJoinRule and
ProjectJoinTransposeRule but i was unable to repro the above issue in
LoptOptimizeJoinRule and after some research, i found out that this issue has
been fixed already( we need to cherry-pick this change). See
[this|https://issues.apache.org/jira/browse/CALCITE-2720].
Since we have already fixed the issue of empty aggregate in
LoptOptimizeJoinRule, i think we should also fix the same for
ProjectJoinTransposeRule. I could not create the query which would fail for
ProjectJoinTransposeRule since we do a lot of transformations, but this simple
test will repro the issue:
{code:java}
@Test void testProjectJoinTransposeRuleOnAggWithNoFields() {
final RelBuilder relBuilder =
RelBuilder.create(RelBuilderTest.config().build());
// Build a rel equivalent to sql:
// SELECT name FROM (SELECT count(*) cnt_star, count(empno) cnt_en FROM
sales.emp)
// cross join sales.dept
// limit 10
RelNode left = relBuilder.scan("DEPT").build();
RelNode right = relBuilder.scan("EMP")
.project(ImmutableList.of(relBuilder.getRexBuilder().makeExactLiteral(BigDecimal.ZERO)),
ImmutableList.of("DUMMY"))
.aggregate(relBuilder.groupKey()).build();
RelNode plan = relBuilder.push(left)
.push(right)
.join(JoinRelType.INNER,
relBuilder.getRexBuilder().makeLiteral(true))
.project(relBuilder.field("DEPTNO"))
.build();
HepProgram program = new HepProgramBuilder()
.addRuleInstance(CoreRules.PROJECT_JOIN_TRANSPOSE)
.build();
HepPlanner hepPlanner = new HepPlanner(program);
hepPlanner.setRoot(plan);
hepPlanner.findBestExp();
}
{code}
Regardless, we should fix the RelFieldTrimmer as well, as we discussed above.
> Some rules fail to handle Aggregate node if RelFieldTrimmer trims all the
> fields
> --------------------------------------------------------------------------------
>
> Key: CALCITE-4317
> URL: https://issues.apache.org/jira/browse/CALCITE-4317
> Project: Calcite
> Issue Type: Bug
> Components: core
> Reporter: Rafay A
> Priority: Major
>
> Consider this query:
> {code:java}
> select o_orderkey from
> (SELECT count(*) cnt_star, count(l_orderkey) cnt_ok FROM lineitem)
> cross join orders
> limit 10
> {code}
> which generates this plan:
> {code:java}
> LogicalSort(fetch=[10])
> LogicalProject(o_orderkey=[$2])
> LogicalJoin(condition=[true], joinType=[inner])
> LogicalAggregate(group=[{}], cnt_star=[COUNT()], cnt_ok=[COUNT($0)])
> LogicalProject(l_orderkey=[$0])
> ScanCrel(table=[lineitem], columns=[`l_orderkey`, `l_partkey`,
> `l_suppkey`, `l_linenumber`, `l_quantity`, `l_extendedprice`, `l_discount`,
> `l_tax`, `l_returnflag`, `l_linestatus`, `l_shipdate`, `l_commitdate`,
> `l_receiptdate`, `l_shipinstruct`, `l_shipmode`, `l_comment`])
> ScanCrel(table=[orders], columns=[`o_orderkey`, `o_custkey`,
> `o_orderstatus`, `o_totalprice`, `o_orderdate`, `o_orderpriority`, `o_clerk`,
> `o_shippriority`, `o_comment`])
> {code}
> and after we apply RelFieldTrimmer:
> {code:java}
> LogicalSort(fetch=[10])
> LogicalProject(o_orderkey=[$0])
> LogicalJoin(condition=[true], joinType=[inner])
> LogicalAggregate(group=[{}])
> LogicalProject(l_orderkey=[$0])
> ScanCrel(table=[lineitem], columns=[`l_orderkey`])
> ScanCrel(table=[orders], columns=[`o_orderkey`])
> {code}
>
> We see that the LogicalAggregate has no groups/functions after applying the
> trimmer. When we apply ProjectJoinTransposeRule or LoptOptimizeJoinRule after
> trimming, we are seeing these exceptions:
> For LoptOptimizeJoinRule:
> {code:java}
> (java.lang.ArrayIndexOutOfBoundsException) 0
> com.google.common.collect.RegularImmutableList.get():75
> org.apache.calcite.rel.metadata.RelMdColumnOrigins.getColumnOrigins():77
> sun.reflect.NativeMethodAccessorImpl.invoke0():-2
> sun.reflect.NativeMethodAccessorImpl.invoke():62
> sun.reflect.DelegatingMethodAccessorImpl.invoke():43
> java.lang.reflect.Method.invoke():498
>
> org.apache.calcite.rel.metadata.ReflectiveRelMetadataProvider$1$1.invoke():178
> com.sun.proxy.$Proxy175.getColumnOrigins():-1
> sun.reflect.GeneratedMethodAccessor144.invoke():-1
> sun.reflect.DelegatingMethodAccessorImpl.invoke():43
> java.lang.reflect.Method.invoke():498
>
> org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke():139
> com.sun.proxy.$Proxy175.getColumnOrigins():-1
> sun.reflect.GeneratedMethodAccessor144.invoke():-1
> sun.reflect.DelegatingMethodAccessorImpl.invoke():43
> java.lang.reflect.Method.invoke():498
> com.sun.proxy.$Proxy175.getColumnOrigins():-1
> sun.reflect.GeneratedMethodAccessor144.invoke():-1
> sun.reflect.DelegatingMethodAccessorImpl.invoke():43
> java.lang.reflect.Method.invoke():498
>
> org.apache.calcite.rel.metadata.ChainedRelMetadataProvider$ChainedInvocationHandler.invoke():139
> com.sun.proxy.$Proxy175.getColumnOrigins():-1
> sun.reflect.GeneratedMethodAccessor144.invoke():-1
> sun.reflect.DelegatingMethodAccessorImpl.invoke():43
> java.lang.reflect.Method.invoke():498
> com.sun.proxy.$Proxy175.getColumnOrigins():-1
> sun.reflect.GeneratedMethodAccessor144.invoke():-1
> sun.reflect.DelegatingMethodAccessorImpl.invoke():43
> java.lang.reflect.Method.invoke():498
>
> org.apache.calcite.rel.metadata.RelMetadataQuery$MetadataInvocationHandler.invoke():110
> com.sun.proxy.$Proxy175.getColumnOrigins():-1
> org.apache.calcite.rel.metadata.RelMetadataQuery.getColumnOrigins():293
> org.apache.calcite.rel.metadata.RelMetadataQuery.getTableOrigin():348
> org.apache.calcite.rel.rules.LoptOptimizeJoinRule.getSimpleFactors():377
>
> org.apache.calcite.rel.rules.LoptOptimizeJoinRule.findRemovableSelfJoins():297
> org.apache.calcite.rel.rules.LoptOptimizeJoinRule.onMatch():126
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule():317
> org.apache.calcite.plan.hep.HepPlanner.applyRule():556
> org.apache.calcite.plan.hep.HepPlanner.applyRules():415
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction():280
> org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute():74
> org.apache.calcite.plan.hep.HepPlanner.executeProgram():211
> org.apache.calcite.plan.hep.HepPlanner.findBestExp():198
> {code}
> For ProjectJoinTransposeRule:
> {code:java}
> (java.lang.ArrayIndexOutOfBoundsException) 0
> com.google.common.collect.RegularImmutableList.get():75
> org.apache.calcite.rel.rules.PushProjector.createProjectRefsAndExprs():477
> org.apache.calcite.rel.rules.ProjectJoinTransposeRule.onMatch():138
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule():317
> org.apache.calcite.plan.hep.HepPlanner.applyRule():556
> org.apache.calcite.plan.hep.HepPlanner.applyRules():415
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction():280
> org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute():74
> org.apache.calcite.plan.hep.HepPlanner.executeProgram():211
> org.apache.calcite.plan.hep.HepPlanner.findBestExp():198
> {code}
> I think we should either fix the trimmer, or the rules to handle these cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
