Rahul Challapalli created DRILL-4832:
----------------------------------------
Summary: Metadata Cache Pruning is not taking place in some rare
scenarios
Key: DRILL-4832
URL: https://issues.apache.org/jira/browse/DRILL-4832
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Reporter: Rahul Challapalli
git.commit.id.abbrev=f476eb5
The below set of queries is one test case. I usually run 10 instances of the
same test case in parallel and we read the cache file from the appropriate
sub-directory in almost all the cases.
But only once out of 25 such runs, I observed that we are reading the cache
from the root level instead of the sub-directory for the "l_3level" table.
{code}
refresh table metadata l_3level;
refresh table metadata c_1level;
refresh table metadata o_2level;
explain plan for select
l.l_orderkey,
sum(l.l_extendedprice * (1 - l.l_discount)) as revenue,
o.o_orderdate,
o.o_shippriority
from
c_1level c,
o_2level o,
l_3level l
where
c.c_mktsegment = 'HOUSEHOLD'
and c.c_custkey = o.o_custkey
and l.l_orderkey = o.o_orderkey
and l.dir0 = 1 and l.dir1 = 'three' and l.dir2 = '2015-7-12'
and o.dir0 = '1991' and o.dir1 = 'feb'
and o.o_orderdate < date '1995-03-25'
and l.l_shipdate > date '1995-03-25'
group by
l.l_orderkey,
o.o_orderdate,
o.o_shippriority
00-00 Screen
00-01 Project(l_orderkey=[$0], o_orderdate=[$1], o_shippriority=[$2])
00-02 Project(l_orderkey=[$8], o_orderdate=[$6], o_shippriority=[$7])
00-03 HashJoin(condition=[=($1, $2)], joinType=[inner])
00-05 SelectionVectorRemover
00-08 Filter(condition=[=($0, 'HOUSEHOLD')])
00-11 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=/drill/testdata/metadata_caching_pp/c_1level/1991/customer.parquet],
ReadEntryWithPath
[path=/drill/testdata/metadata_caching_pp/c_1level/1992/customer.parquet]],
selectionRoot=/drill/testdata/metadata_caching_pp/c_1level, numFiles=2,
usedMetadataFile=true,
cacheFileRoot=/drill/testdata/metadata_caching_pp/c_1level,
columns=[`c_mktsegment`, `c_custkey`]]])
00-04 HashJoin(condition=[=($6, $1)], joinType=[inner])
00-07 SelectionVectorRemover
00-10 Filter(condition=[AND(=($2, '1991'), =($3, 'feb'), <($4,
1995-03-25))])
00-13 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=/drill/testdata/metadata_caching_pp/o_2level/1991/feb/orders.parquet]],
selectionRoot=/drill/testdata/metadata_caching_pp/o_2level, numFiles=1,
usedMetadataFile=true,
cacheFileRoot=/drill/testdata/metadata_caching_pp/o_2level/1991/feb,
columns=[`o_custkey`, `o_orderkey`, `dir0`, `dir1`, `o_orderdate`,
`o_shippriority`]]])
00-06 Project(l_orderkey=[$0], dir00=[$1], dir10=[$2], dir2=[$3],
l_shipdate=[$4])
00-09 SelectionVectorRemover
00-12 Filter(condition=[AND(=($1, 1), =($2, 'three'), =($3,
'2015-7-12'), >($4, 1995-03-25))])
00-14 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath
[path=/drill/testdata/metadata_caching_pp/l_3level/1/three/2015-7-12/40.parquet]],
selectionRoot=/drill/testdata/metadata_caching_pp/l_3level, numFiles=1,
usedMetadataFile=true,
cacheFileRoot=/drill/testdata/metadata_caching_pp/l_3level,
columns=[`l_orderkey`, `dir0`, `dir1`, `dir2`, `l_shipdate`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
