[
https://issues.apache.org/jira/browse/IMPALA-12598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17822323#comment-17822323
]
ASF subversion and git services commented on IMPALA-12598:
----------------------------------------------------------
Commit 65094a74f1730f67b19505ffacc1db1ebda98cee in impala's branch
refs/heads/master from Gabor Kaszab
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=65094a74f ]
IMPALA-12598: Allow multiple equality field id lists for Iceberg tables
This patch adds support for reading Iceberg tables that have
different equality field ID lists associated to different equality
delete files. In practice this is a use case when one equality delete
file deletes by e.g. columnA and columnB while another one deletes by
columnB and columnC.
In order to achieve such functionality the plan tree creation needed
some adjustments so that it can create separate LEFT ANTI JOIN nodes
for the different equality field ID lists.
Testing:
- Flink and NiFi was used for creating some test tables with the
desired equality field IDs. Coverage on these tables are added to
the test suite.
Change-Id: I3e52d7a5800bf1b479f0c234679be92442d09f79
Reviewed-on: http://gerrit.cloudera.org:8080/20951
Reviewed-by: Gabor Kaszab <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add support for multiple equality field ID list
> -----------------------------------------------
>
> Key: IMPALA-12598
> URL: https://issues.apache.org/jira/browse/IMPALA-12598
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Gabor Kaszab
> Assignee: Gabor Kaszab
> Priority: Major
> Labels: impala-iceberg
>
> Iceberg metadata holds an equality field ID list for the equality-delete
> files. It's possible to have a different equality field ID list for different
> equality-delete files, for instance one file deletes by columnA while another
> file deletes by columnB.
> When you have such a table you should have multiple layers of ANTI JOINs, one
> join for each equality field ID list.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
