[
https://issues.apache.org/jira/browse/IMPALA-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18098771#comment-18098771
]
ASF subversion and git services commented on IMPALA-15197:
----------------------------------------------------------
Commit f435772b22ed1be2f9c6ae84d954e4bfd29e7c4c in impala's branch
refs/heads/master from Peter Rozsa
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f435772b2 ]
IMPALA-15211: Fix file count for iceberg_lineitem_sixblocks in test_web_pages.py
IMPALA-15197 introduced shuffling by file path for unpartitioned
Iceberg V2 tables as well, it caused iceberg_lineitem_sixblocks to
contain only 2 files (1 data, 1 delete), and it broke test_web_pages.py.
This patch fixes the assertion on the number of files.
Change-Id: I1c09c6a127d1324272dd18f500756d44a7ee6a25
Reviewed-on: http://gerrit.cloudera.org:8080/24628
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Iceberg V2 UPDATE with join can miss duplicate-row detection in distributed
> execution
> -------------------------------------------------------------------------------------
>
> Key: IMPALA-15197
> URL: https://issues.apache.org/jira/browse/IMPALA-15197
> Project: IMPALA
> Issue Type: Bug
> Reporter: Peter Rozsa
> Assignee: Peter Rozsa
> Priority: Major
> Labels: impala-iceberg
> Fix For: Impala 5.0.0
>
>
> UPDATE ... FROM on an unpartitioned Iceberg V2 table that joins against a
> reference table may not detect when the same target row matches multiple
> reference rows. In that case the UPDATE should fail with Duplicated row in
> DELETE sink. but instead succeeds silently and produces a duplicate row in
> the table.
>
> {code:java}
> create table tgt (id int, val bigint)
> stored by iceberg tblproperties ('format-version'='2');
> insert into tgt values (0,0),(1,10),(2,20),(3,30),(4,40),(5,50),(6,60),(7,70);
> compute stats tgt;
> -- Reference table with no stats so the planner puts it on the probe side
> create table ref_pq (i int, bi bigint) stored as parquet;
> insert into ref_pq values
> (0,100),(1,101),(2,102),(3,103),(4,104),(5,105),(6,106),(7,107);
> insert into ref_pq values
> (0,200),(1,201),(2,202),(3,203),(4,204),(5,205),(6,206),(7,207);
> -- Each id in ref_pq appears twice -> multiple matches per target row
> update tgt set val = bi + 1 from tgt, ref_pq where id = i;
> select count(*) total, count(distinct id) distinct_ids from tgt; {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
