[
https://issues.apache.org/jira/browse/CALCITE-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16342458#comment-16342458
]
Amruth S commented on CALCITE-1868:
-----------------------------------
RelDataTypeFactoryImpl's LoadingCache seem to create Fields with indices based
on the current row ordering.
But in AggregateProjectPullUpConstantsRule, the navigable map that stores the
constants seem to hash it against the original ordinal. So when iterating over
the fieldlist of aggregate row types to create, to create the projection again,
wrong datatypes are getting mapped against the column names.
In this particular example shared, the original type is a(bigint), c(integer),
d(varchar). But post the transformation the type ends up as a(bigint),
c(varchar), d(integer) failing in type equivalence.
> AssertionError in SELECT DISTINCT subquery
> ------------------------------------------
>
> Key: CALCITE-1868
> URL: https://issues.apache.org/jira/browse/CALCITE-1868
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.6.0
> Environment: Hive 2.1
> calcite 1.6.0.
> Reporter: Artem Velykorodnyi
> Assignee: Julian Hyde
> Priority: Major
> Labels: cbo, hive
>
> Trying to execute the query on hive 2.1.1 its failed with exception.
> Query:
> {code}
> SELECT
> account_id,
> subscription_id,
> previous_subscription_id,
> subscription_status,
> subscription_effective_from_dt,
> subscription_effective_to_dt
> FROM
> (
> SELECT
> DISTINCT account_id,
> subscription_id,
> previous_subscription_id,
> subscription_status,
> subscription_effective_from_dt,
> subscription_effective_to_dt,
> order_completed_dtm,
> ROW_NUMBER()OVER(PARTITION BY ACCOUNT_ID) as r1
> FROM temp.i
> WHERE account_id=516272263
> ) AS RTE where r1=1;
> {code}
> If I disable CBO, query works fine
> hive> set hive.cbo.enable=false;
> Also it works when remove DISTINCT clause from sub-SELECT.
> And below query works too:
> {code}
> SELECT
> DISTINCT account_id,
> subscription_id,
> previous_subscription_id,
> subscription_status,
> subscription_effective_from_dt,
> subscription_effective_to_dt,
> order_completed_dtm,
> ROW_NUMBER()OVER(PARTITION BY ACCOUNT_ID) as r1
> FROM temp.i
> WHERE account_id=516272263;
> {code}
> Exception:
> {code}
> Exception in thread "main" java.lang.AssertionError: Internal error: Cannot
> add expression of different type to set:
> set type is RecordType(BIGINT subscription_id, BIGINT
> previous_subscription_id, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" subscription_status, DATE
> subscription_effective_from_dt, DATE subscription_effective_to_dt,
> TIMESTAMP(9) order_completed_dtm, INTEGER r1) NOT NULL
> expression type is RecordType(BIGINT subscription_id, BIGINT
> previous_subscription_id, VARCHAR(2147483647) CHARACTER SET "UTF-16LE"
> COLLATE "ISO-8859-1$en_US$primary" subscription_status, DATE
> subscription_effective_from_dt, DATE subscription_effective_to_dt, INTEGER
> order_completed_dtm, TIMESTAMP(9) r1) NOT NULL
> set is rel#73:HiveAggregate.HIVE.[](input=HepRelVertex#82,group={1, 2, 3, 4,
> 5, 6, 7})
> expression is HiveProject#84
> at org.apache.calcite.util.Util.newInternal(Util.java:774)
> at
> org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:317)
> at
> org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:57)
> at
> org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:224)
> at
> org.apache.calcite.rel.rules.AggregateProjectPullUpConstantsRule.onMatch(AggregateProjectPullUpConstantsRule.java:227)
> at
> org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:318)
> at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:514)
> at
> org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:392)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:285)
> at
> org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:72)
> at
> org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:207)
> at
> org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:194)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:1318)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:1189)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:970)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:901)
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:969)
> at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149)
> at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:719)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:287)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10887)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:246)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:478)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1243)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1385)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1172)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1162)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:234)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:185)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:401)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:791)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:729)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:652)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:647)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
