Aman Sinha created DRILL-4071:
---------------------------------
Summary: Partition pruning fails when a Coalesce() function
appears with partition filter
Key: DRILL-4071
URL: https://issues.apache.org/jira/browse/DRILL-4071
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Reporter: Aman Sinha
Pruning fails for this query:
{code}
0: jdbc:drill:zk=local> explain plan for select count(*) from
dfs.`/Users/asinha/data/multilevel/parquet` where dir0 = 1994 and
coalesce(o_clerk, 'Clerk') = '';
+------+------+
| text | json |
+------+------+
| 00-00 Screen
00-01 Project(EXPR$0=[$0])
00-02 StreamAgg(group=[{}], EXPR$0=[COUNT()])
00-03 Project($f0=[0])
00-04 SelectionVectorRemover
00-05 Filter(condition=[AND(=($0, 1994), =(CASE(IS NOT NULL($1),
$1, 'Clerk'), ''))])
00-06 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1994/Q1/orders_94_q1.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1994/Q2/orders_94_q2.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1994/Q3/orders_94_q3.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1994/Q4/orders_94_q4.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1995/Q1/orders_95_q1.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1995/Q2/orders_95_q2.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1995/Q3/orders_95_q3.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1995/Q4/orders_95_q4.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1996/Q1/orders_96_q1.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1996/Q2/orders_96_q2.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1996/Q3/orders_96_q3.parquet],
ReadEntryWithPath
[path=file:/Users/asinha/data/multilevel/parquet/1996/Q4/orders_96_q4.parquet]],
selectionRoot=file:/Users/asinha/data/multilevel/parquet, numFiles=12,
usedMetadataFile=false, columns=[`dir0`, `o_clerk`]]])
{code}
The log indicates no partition filters were found:
{code}
...
o.a.d.e.p.l.partition.PruneScanRule - No conditions were found eligible for
partition pruning.Total pruning elapsed time: 0 ms
{code}
A preliminary analysis indicates that since the Coalesce gets converted to a
CASE(IS NOT NULL....) expression, the filter analysis does not correctly
process the full expression tree. At one point in
{{FindPartitionConditions.analyzeCall()}} I saw the operandStack had 3 elements
in it: [NO_PUSH, NO_PUSH, PUSH] which seemed strange since I would expect even
number of elements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
