Boaz Ben-Zvi created DRILL-7012:
-----------------------------------
Summary: Make SelectionVectorRemover project only the needed
columns
Key: DRILL-7012
URL: https://issues.apache.org/jira/browse/DRILL-7012
Project: Apache Drill
Issue Type: Improvement
Components: Execution - Relational Operators, Query Planning &
Optimization
Affects Versions: 1.15.0
Reporter: Boaz Ben-Zvi
A SelectionVectorRemover is often used after a filter, to copy into a newly
allocated new batch only the "filtered out" rows. In some cases the columns
used by the filter are not needed downstream; currently these columns are being
needlessly allocated and copied, and later removed by a Project.
_+Suggested improvement+_: The planner can pass the information about these
columns to the SelectionVectorRemover, which would avoid this useless
allocation and copy. The Planner would also eliminate that Project from the
plan.
Here is an example, the query:
{code:java}
select max(l_quantity) from cp.`tpch/lineitem.parquet` L where L.l_orderkey >
58999 and L.l_shipmode = 'TRUCK' group by l_linenumber ;
{code}
And the result plan (trimmed for readability), where "l_orderkey" and
"l_shipmode" are removed by the Project:
{noformat}
00-00 Screen : rowType = RecordType(ANY EXPR$0):
00-01 Project(EXPR$0=[$0]) : rowType = RecordType(ANY EXPR$0):
00-02 Project(EXPR$0=[$1]) : rowType = RecordType(ANY EXPR$0):
00-03 HashAgg(group=[\{0}], EXPR$0=[MAX($1)]) : rowType = RecordType(ANY
l_linenumber, ANY EXPR$0):
00-04 *Project*(l_linenumber=[$2], l_quantity=[$3]) : rowType = RecordType(ANY
l_linenumber, ANY l_quantity):
00-05 *SelectionVectorRemover* : rowType = RecordType(ANY *l_orderkey*, ANY
*l_shipmode*, ANY l_linenumber, ANY l_quantity):
00-06 *Filter*(condition=[AND(>($0, 58999), =($1, 'TRUCK'))]) : rowType =
RecordType(ANY l_orderkey, ANY l_shipmode, ANY l_linenumber, ANY l_quantity):
00-07 Scan(table=[[cp, tpch/lineitem.parquet]], groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=classpath:/tpch/lineitem.parquet]],
selectionRoot=classpath:/tpch/lineitem.parquet, numFiles=1, numRowGroups=1,
usedMetadataFile=false, columns=[`l_orderkey`, `l_shipmode`, `l_linenumber`,
`l_quantity`]]]) : rowType = RecordType(ANY l_orderkey, ANY l_shipmode, ANY
l_linenumber, ANY l_quantity):
{noformat}
The implementation will not be simple, as the relevant code (e.g.,
GenericSV2Copier) has no idea of specific columns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
