Gabor Kaszab created IMPALA-12599:
-------------------------------------
Summary: Support equality delete files that don't contain the
partition values
Key: IMPALA-12599
URL: https://issues.apache.org/jira/browse/IMPALA-12599
Project: IMPALA
Issue Type: Sub-task
Components: Frontend
Reporter: Gabor Kaszab
When you write equality delete files with Flink the partition columns have to
also be part of the primary key. As a result the partition values will be added
into the equality delete files. However, the Iceberg spec is more flexible than
that and it's also a valid case when the partition values aren't written into
the eq-delete files.
To be able to read such tables Impala should also check if the partition spec
and the partition values match between the data and delete files when applying
the delete rows. This could be achieved by adding a virtual columns and
conjuncts for the partition spec IDs and also for the partition values. These
virtual columns already exist, but have to be added to the scan nodes, and the
conjuncts have to be created for the ANTI JOIN node.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
