[
https://issues.apache.org/jira/browse/HIVE-24072?focusedWorklogId=479613&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-479613
]
ASF GitHub Bot logged work on HIVE-24072:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Sep/20 11:56
Start Date: 07/Sep/20 11:56
Worklog Time Spent: 10m
Work Description: kgyrtkirk commented on a change in pull request #1432:
URL: https://github.com/apache/hive/pull/1432#discussion_r484385748
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveAggregateJoinTransposeRule.java
##########
@@ -145,8 +145,7 @@ public void onMatch(RelOptRuleCall call) {
int fieldCount = joinInput.getRowType().getFieldCount();
final ImmutableBitSet fieldSet =
ImmutableBitSet.range(offset, offset + fieldCount);
- final ImmutableBitSet belowAggregateKeyNotShifted =
- belowAggregateColumns.intersect(fieldSet);
Review comment:
right; the column which was causing the trouble was the preceeding
join's joinkey.
the issue was caused by
* that between the previous / and current join there were no projects - so
all the join keys of the previous join were present in the input
* meanwhile the aggregate had references to `0,2,...` columns - which were
unique; so the logic assumed that the `joinInput` could be used as is
however..because column 1 was present in the input; but not in the output
this have caused that the actual join was not in sync with the mapping being
created
I think the patch may make more sense than the above reasoning :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 479613)
Time Spent: 50m (was: 40m)
> HiveAggregateJoinTransposeRule may try to create an invalid transformation
> --------------------------------------------------------------------------
>
> Key: HIVE-24072
> URL: https://issues.apache.org/jira/browse/HIVE-24072
> Project: Hive
> Issue Type: Bug
> Reporter: Zoltan Haindrich
> Assignee: Zoltan Haindrich
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {code}
> java.lang.AssertionError:
> Cannot add expression of different type to set:
> set type is RecordType(INTEGER NOT NULL o_orderkey, DECIMAL(10, 0)
> o_totalprice, DATE o_orderdate, INTEGER NOT NULL c_custkey, VARCHAR(25)
> CHARACTER SET "UTF-16LE" c_name, DOUBLE $f5) NOT NULL
> expression type is RecordType(INTEGER NOT NULL o_orderkey, INTEGER NOT NULL
> o_custkey, DECIMAL(10, 0) o_totalprice, DATE o_orderdate, INTEGER NOT NULL
> c_custkey, DOUBLE $f1) NOT NULL
> set is rel#567:HiveAggregate.HIVE.[].any(input=HepRelVertex#490,group={2, 4,
> 5, 6, 7},agg#0=sum($1))
> expression is HiveProject(o_orderkey=[$2], o_custkey=[$3], o_totalprice=[$4],
> o_orderdate=[$5], c_custkey=[$6], $f1=[$1])
> HiveJoin(condition=[=($2, $0)], joinType=[inner], algorithm=[none],
> cost=[{2284.5 rows, 0.0 cpu, 0.0 io}])
> HiveAggregate(group=[{0}], agg#0=[sum($1)])
> HiveProject(l_orderkey=[$0], l_quantity=[$4])
> HiveTableScan(table=[[tpch_0_001, lineitem]], table:alias=[l])
> HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none],
> cost=[{1.0000000000009115E15 rows, 0.0 cpu, 0.0 io}])
> HiveJoin(condition=[=($4, $1)], joinType=[inner], algorithm=[none],
> cost=[{1650.0 rows, 0.0 cpu, 0.0 io}])
> HiveProject(o_orderkey=[$0], o_custkey=[$1], o_totalprice=[$3],
> o_orderdate=[$4])
> HiveTableScan(table=[[tpch_0_001, orders]], table:alias=[orders])
> HiveProject(c_custkey=[$0], c_name=[$1])
> HiveTableScan(table=[[tpch_0_001, customer]],
> table:alias=[customer])
> HiveProject($f0=[$0])
> HiveFilter(condition=[>($1, 3E2)])
> HiveAggregate(group=[{0}], agg#0=[sum($4)])
> HiveTableScan(table=[[tpch_0_001, lineitem]],
> table:alias=[lineitem])
> at
> org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:383)
> at
> org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:57)
> at
> org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:236)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveAggregateJoinTransposeRule.onMatch(HiveAggregateJoinTransposeRule.java:300)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
