Rahul Challapalli created DRILL-4860:
----------------------------------------
Summary: Partition pruning does not work when we have a correlated
sub-query
Key: DRILL-4860
URL: https://issues.apache.org/jira/browse/DRILL-4860
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Affects Versions: 1.8.0
Reporter: Rahul Challapalli
git commit # : 0a4c21cc15329c063f256f6fbf2c4c69a90d9fa1
The below query indicates that partition pruning did not occur for the sub-query
{code}
explain plan for select count (*) from (
select t1.l_orderkey, t1.l_linenumber from
dfs.`/drill/testdata/partition_pruning/nested/l_3level` t1
where
t1.dir0 = 1 and t1.dir1='one' and t1.dir2 = '2015-7-12' and t1.l_orderkey
in (
select t2.l_orderkey from
dfs.`/drill/testdata/partition_pruning/nested/l_3level` t2 where t2.dir0 = 1
and t2.dir1='two' and t2.dir2 = '2015-8-12' and t2.l_discount > t1.l_discount
)
) data;
+------+------+
| text | json |
+------+------+
| 00-00 Screen
00-01 Project(EXPR$0=[$0])
00-02 StreamAgg(group=[{}], EXPR$0=[COUNT()])
00-03 Project($f0=[0])
00-04 Project(l_discount=[$2], l_orderkey=[$3], l_orderkey0=[$0],
l_discount0=[$1])
00-05 HashJoin(condition=[AND(=($2, $1), =($3, $0))],
joinType=[inner])
00-07 HashAgg(group=[{0, 1}])
00-09 Project(l_orderkey=[$0], l_discount0=[$2])
00-10 Project(l_orderkey=[$1], l_discount=[$2],
l_discount0=[$0])
00-11 HashJoin(condition=[=($2, $0)], joinType=[inner])
00-13 HashAgg(group=[{0}])
00-15 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/1/two/2015-8-12/30.parquet],
ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/1/three/2015-7-12/40.parquet],
ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/1/one/2015-7-13/20.parquet],
ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/1/one/2015-7-12/10.parquet],
ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/2/two/2015-9-12/50.parquet],
ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/2/two/2015-7-12/30.parquet],
ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/2/two/2015-8-12/40.parquet],
ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/2/three/2015-8-12/10.parquet],
ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/2/one/2015-9-12/20.parquet],
ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/2/one/2015-7-12/50.parquet],
ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/2/one/2015-8-12/10.parquet]],
selectionRoot=maprfs:/drill/testdata/partition_pruning/nested/l_3level,
numFiles=11, usedMetadataFile=false, columns=[`l_discount`]]])
00-12 Project(l_orderkey=[$0], l_discount0=[$1])
00-14 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/1/two/2015-8-12/30.parquet]],
selectionRoot=maprfs:/drill/testdata/partition_pruning/nested/l_3level,
numFiles=1, usedMetadataFile=false, columns=[`l_orderkey`, `l_discount`]]])
00-06 Project(l_discount=[$0], l_orderkey0=[$1])
00-08 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=maprfs:/drill/testdata/partition_pruning/nested/l_3level/1/one/2015-7-12/10.parquet]],
selectionRoot=maprfs:/drill/testdata/partition_pruning/nested/l_3level,
numFiles=1, usedMetadataFile=false, columns=[`l_discount`, `l_orderkey`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
