Jinfeng Ni created DRILL-3591:
---------------------------------
Summary: Partition pruning hit AssertionError when a cast function
is used in comparison operand
Key: DRILL-3591
URL: https://issues.apache.org/jira/browse/DRILL-3591
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Affects Versions: 1.1.0
Reporter: Jinfeng Ni
Assignee: Aman Sinha
In Drill unit test TestPartitionFilter.testPartitionFilter6_Parquet(), the
query is :
{code}
select * from dfs_test.tmp.parquet where (yr=1995 and o_totalprice < 40000) or
yr=1996
{code}
If I slightly modify the filter, by adding a cast function: {
{code}
select * from dfs_test.`%s/multilevel/parquet` where (dir0=cast(1995 as
varchar(10)) and o_totalprice < 40000) or dir0=1996
{code}
It will hit Assertion Error, when PruneScanRule calls interpreter to evaluate
the filter condition.
{code}
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: AssertionError
...
Caused by: java.lang.AssertionError
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.isBitOn(InterpreterEvaluator.java:490)
~[classes/:na]
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanAnd(InterpreterEvaluator.java:434)
~[classes/:na]
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanOperator(InterpreterEvaluator.java:332)
~[classes/:na]
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanOperator(InterpreterEvaluator.java:147)
~[classes/:na]
at
org.apache.drill.common.expression.BooleanOperator.accept(BooleanOperator.java:36)
~[classes/:na]
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanOr(InterpreterEvaluator.java:463)
~[classes/:na]
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanOperator(InterpreterEvaluator.java:334)
~[classes/:na]
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitBooleanOperator(InterpreterEvaluator.java:147)
~[classes/:na]
at
org.apache.drill.common.expression.BooleanOperator.accept(BooleanOperator.java:36)
~[classes/:na]
at
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator.evaluate(InterpreterEvaluator.java:80)
~[classes/:na]
at
org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:420)
~[classes/:na]
at
org.apache.drill.exec.planner.logical.partition.PruneScanRule$2.onMatch(PruneScanRule.java:156)
~[classes/:na]
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
~[calcite-core-1.1.0-drill-r15.jar:1.1.0-drill-r15]
at
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:795)
~[calcite-core-1.1.0-drill-r15.jar:1.1.0-drill-r15]
at
org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303)
~[calcite-core-1.1.0-drill-r15.jar:1.1.0-drill-r15]
at
org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:316)
~[calcite-core-1.1.0-drill-r15.jar:1.1.0-drill-r15]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.logicalPlanningVolcanoAndLopt(DefaultSqlHandler.java:528)
~[classes/:na]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:213)
~[classes/:na]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:248)
~[classes/:na]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:164)
~[classes/:na]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178)
~[classes/:na]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:903)
[classes/:na]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:242)
[classes/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
... 13 more
{code}
I debug a bit, seems in PartitionPruneRule, the condition is rewrote to the
following, which seems not right, since CAST function becomes one operand of
"AND".
{code}
OR(AND(CAST(1995):VARCHAR(10) CHARACTER SET "ISO-8859-1" COLLATE
"ISO-8859-1$en_US$primary" NOT NULL, =($1, CAST(1995):VARCHAR(10) CHARACTER SET
"ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary" NOT NULL)), =($1, 1996))
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
