Jinfeng Ni created DRILL-5359:
---------------------------------
Summary: ClassCastException when push down filter on the output of
flatten into parquet scan
Key: DRILL-5359
URL: https://issues.apache.org/jira/browse/DRILL-5359
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Affects Versions: 1.9.0
Reporter: Jinfeng Ni
Assignee: Jinfeng Ni
Fix For: 1.11.0
The following simplified query would hit ClassCastException.
{code}
select n_regionkey
from (select n_regionkey,
flatten(nation.cities) as cities
from cp.`tpch/nation.parquet` nation) as flattenedCities
where flattenedCities.cities.`zip` = '12345';
{code}
Here is the stacktrace for the Exception :
{code}
caused by: java.lang.ClassCastException:
org.apache.drill.common.expression.FunctionCall cannot be cast to
org.apache.drill.common.expression.SchemaPath
at
org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:170)
~[classes/:na]
at
org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:80)
~[classes/:na]
at org.apache.calcite.rex.RexCall.accept(RexCall.java:107)
~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
at
org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.doFunction(DrillOptiq.java:205)
~[classes/:na]
at
org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:105)
~[classes/:na]
at
org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall(DrillOptiq.java:80)
~[classes/:na]
at org.apache.calcite.rex.RexCall.accept(RexCall.java:107)
~[calcite-core-1.4.0-drill-r19.jar:1.4.0-drill-r19]
at
org.apache.drill.exec.planner.logical.DrillOptiq.toDrill(DrillOptiq.java:77)
~[classes/:na]
at
org.apache.drill.exec.store.parquet.ParquetPushDownFilter.doOnMatch(ParquetPushDownFilter.java:141)
~[classes/:na]
at
org.apache.drill.exec.store.parquet.ParquetPushDownFilter$1.onMatch(ParquetPushDownFilter.java:68)
~[classes/:na]
{code}
The cause of this problem: Parquet filter pushdown rule tries to push a filter
expression containing item/flatten operators into parquet scan. However, the
method DrillOptiq.toDrill() does not allow such expression (since "flatten" is
not a scalar function).
The solution is to disable pushing such filter expression. Even the rule
allows, the underneath parquet metadata would not have the corresponding
statistics; there is no point to considering push such filter expression.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
