Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/24733 )
Change subject: IMPALA-14996: Optimize count(*) for Iceberg V3 DV-only tables ...................................................................... Patch Set 3: (3 comments) http://gerrit.cloudera.org:8080/#/c/24733/3/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java File fe/src/main/java/org/apache/impala/analysis/SelectStmt.java: http://gerrit.cloudera.org:8080/#/c/24733/3/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java@1561 PS3, Line 1561: hasDeleteFiles It's better to check hasDeleteFiles first as hasOnlyDeletionVectors is doing a full scan, which could be avoided http://gerrit.cloudera.org:8080/#/c/24733/3/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java File fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java: http://gerrit.cloudera.org:8080/#/c/24733/3/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java@1258 PS3, Line 1258: getTotalPositionDeletes What happens if the table has dangling deletes? the total pos del count contains it http://gerrit.cloudera.org:8080/#/c/24733/3/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteIcebergTable.java File java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteIcebergTable.java: http://gerrit.cloudera.org:8080/#/c/24733/3/java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteIcebergTable.java@97 PS3, Line 97: getRecordCount() > 0 && (!hasDeleteFiles() || hasOnlyDeletionVectors()); Consider the following example: CREATE DATABASE IF NOT EXISTS repro_i14996; CREATE TABLE repro_i14996.dv_part (i INT, s STRING) PARTITIONED BY SPEC (i) STORED BY ICEBERG TBLPROPERTIES ('format-version'='3'); INSERT INTO repro_i14996.dv_part VALUES (1,'a'), (1,'b'), (1,'c'); INSERT INTO repro_i14996.dv_part VALUES (2,'d'), (2,'e'), (2,'f'); DELETE FROM repro_i14996.dv_part WHERE i = 1 AND s = 'b'; SET planner=ORIGINAL; SELECT count(*) FROM repro_i14996.dv_part; -- 5 SELECT count(*) FROM repro_i14996.dv_part WHERE i > 0; -- 5 SELECT count(*) FROM repro_i14996.dv_part WHERE i = 1; -- 2 SET planner=CALCITE; SET fallback_planner=CALCITE; SELECT count(*) FROM repro_i14996.dv_part; -- 5 correct SELECT count(*) FROM repro_i14996.dv_part WHERE i > 0; -- 2 WRONG (expected 5) SELECT count(*) FROM repro_i14996.dv_part WHERE i = 1; -- 1 WRONG (expected 2) -- To view, visit http://gerrit.cloudera.org:8080/24733 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0e6239629ee779a64808a0a4bbe041ef756c5bb2 Gerrit-Change-Number: 24733 Gerrit-PatchSet: 3 Gerrit-Owner: Arnab Karmakar <[email protected]> Gerrit-Reviewer: Arnab Karmakar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Nandor Kollar <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Comment-Date: Wed, 30 Sep 2026 13:03:09 +0000 Gerrit-HasComments: Yes
