Zoltán Borók-Nagy created IMPALA-15041:
------------------------------------------
Summary: Push down Deletion Vector evaluation to the scanners
Key: IMPALA-15041
URL: https://issues.apache.org/jira/browse/IMPALA-15041
Project: IMPALA
Issue Type: Improvement
Reporter: Zoltán Borók-Nagy
This is an alternative to IMPALA-15040.
To process Iceberg V3 tables we execute the following plan:
{noformat}
UNION ALL
/ \
/ \
/ \
SCAN all IcebergDeleteNode
datafiles / \
without / \
deletes SCAN SCAN
datafiles deletes
with
deletes
{noformat}
IcebergDeleteNode deals with position delete records and Deletion Vectors as
well. Since position delete files are being deprecated by Iceberg V3, in most
cases "SCAN deletes" will be empty, and IcebergDeleteNode only need to deal
with Deletion Vectors.
DV evaluation could be pushed down to "SCAN datafiles with deletes". The would
be especially beneficial in the context of late materialization, in which case
we can skip materializing column values of inactive records.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
