[
https://issues.apache.org/jira/browse/IMPALA-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Rozsa resolved IMPALA-15197.
----------------------------------
Target Version: Impala 5.0.0
Resolution: Fixed
> Iceberg V2 UPDATE with join can miss duplicate-row detection in distributed
> execution
> -------------------------------------------------------------------------------------
>
> Key: IMPALA-15197
> URL: https://issues.apache.org/jira/browse/IMPALA-15197
> Project: IMPALA
> Issue Type: Bug
> Reporter: Peter Rozsa
> Assignee: Peter Rozsa
> Priority: Major
> Labels: impala-iceberg
>
> UPDATE ... FROM on an unpartitioned Iceberg V2 table that joins against a
> reference table may not detect when the same target row matches multiple
> reference rows. In that case the UPDATE should fail with Duplicated row in
> DELETE sink. but instead succeeds silently and produces a duplicate row in
> the table.
>
> {code:java}
> create table tgt (id int, val bigint)
> stored by iceberg tblproperties ('format-version'='2');
> insert into tgt values (0,0),(1,10),(2,20),(3,30),(4,40),(5,50),(6,60),(7,70);
> compute stats tgt;
> -- Reference table with no stats so the planner puts it on the probe side
> create table ref_pq (i int, bi bigint) stored as parquet;
> insert into ref_pq values
> (0,100),(1,101),(2,102),(3,103),(4,104),(5,105),(6,106),(7,107);
> insert into ref_pq values
> (0,200),(1,201),(2,202),(3,203),(4,204),(5,205),(6,206),(7,207);
> -- Each id in ref_pq appears twice -> multiple matches per target row
> update tgt set val = bi + 1 from tgt, ref_pq where id = i;
> select count(*) total, count(distinct id) distinct_ids from tgt; {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
