Soumyakanti Das created HIVE-28564:
--------------------------------------
Summary: AssertionError for jdbc queries when
JDBCExpandExpressionsRule produces non flat Filter conditions
Key: HIVE-28564
URL: https://issues.apache.org/jira/browse/HIVE-28564
Project: Hive
Issue Type: Bug
Security Level: Public (Viewable by anyone)
Components: CBO, JDBC
Affects Versions: 4.1.0
Reporter: Soumyakanti Das
Assignee: Soumyakanti Das
Attachments: jdbc_non_flat_filter_condition.q,
q_test_tpcds_tables_schema.postgres.sql
To repro this, create four tpcds table in postgres by copying
q_test_tpcds_tables_schema.postgres.sql to {{hive/data/scripts/}} directory.
Add jdbc_non_flat_filter_condition.q to {{ql/src/test/queries/clientpositive/}}
and run
{noformat}
mvn test -pl itests/qtest -Pitests -Dtest=TestMiniLlapLocalCliDriver
-Dtest.output.overwrite=true -Dqfile=jdbc_non_flat_filter_condition.q{noformat}
Error stack:
{noformat}
[INFO] Running org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 15.238
s <<< FAILURE! - in org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver
[ERROR]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_non_flat_filter_condition]
Time elapsed: 8.409 s <<< FAILURE!
java.lang.AssertionError: AND(IN($1, 2000, 1999, 2001), OR(=($1, 2000),
OR(AND(=($1, 1999), =($2, 12)), AND(=($1, 2001), =($2, 1)))), IS NOT NULL($0))
at org.apache.calcite.rel.core.Filter.<init>(Filter.java:76)
at
org.apache.calcite.adapter.jdbc.JdbcRules$JdbcFilter.<init>(JdbcRules.java:629)
at
org.apache.calcite.adapter.jdbc.JdbcRules$JdbcFilter.copy(JdbcRules.java:635)
at
org.apache.hadoop.hive.ql.optimizer.calcite.rules.jdbc.JDBCExpandExpressionsRule$FilterCondition.onMatch(JDBCExpandExpressionsRule.java:85)
at
org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:333)
at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:542)
at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:407)
at
org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:271)
at
org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:74)
at
org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:202)
at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:189)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.executeProgram(CalcitePlanner.java:2469)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.executeProgram(CalcitePlanner.java:2428)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPostJoinOrderingTransform(CalcitePlanner.java:2347)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1739)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1573)
at
org.apache.calcite.tools.Frameworks.lambda$withPlanner$0(Frameworks.java:131)
at
org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:914)
at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:180)
at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:126)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1325)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:573)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13164)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:466)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:332)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:180)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:332)
at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:109)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:499)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:451)
at
org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:415){noformat}
In non jdbc Hive queries, the condition is:
{noformat}
AND(IN($6, 2000, 1999, 2001), OR(=($6, 2000), IN(ROW($6, $8), ROW(1999, 12),
ROW(2001, 1)))){noformat}
[JDBCExpandExpressionsRule|https://github.com/apache/hive/blob/6f7c55ab9bc4fd7c3d0c2a6ba3095275b17b3d2d/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/jdbc/JDBCExpandExpressionsRule.java#L209]
converts it to:
{noformat}
AND(IN($1, 2000, 1999, 2001), OR(=($1, 2000), OR(AND(=($1, 1999), =($2, 12)),
AND(=($1, 2001), =($2, 1)))), IS NOT NULL($0)){noformat}
which is not flat as there is an OR within an OR which can be simplified using
{{RexSimplify}} to:
{noformat}
AND(OR(=($1, 2000), AND(=($1, 1999), =($2, 12)), AND(=($1, 2001), =($2, 1))),
IN($1, 2000, 1999, 2001), IS NOT NULL($0)){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
