[
https://issues.apache.org/jira/browse/IMPALA-14993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18089433#comment-18089433
]
ASF subversion and git services commented on IMPALA-14993:
----------------------------------------------------------
Commit 35d64f5ca0a6b7c83fe4785919bcd9aaf19c77d9 in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=35d64f5ca ]
IMPALA-14993: Fix count star optimization incorrectly activating without
count(*)
The Iceberg V2 count star optimization (optimizePlainCountStarQueryV2)
activated for queries like 'SELECT 1 FROM iceberg_table' when the table
had delete files, causing incorrect results (missing rows). The
validation loop rejected non-count(*)/non-constant expressions but never
verified that at least one count(*) was actually present.
Added a hasCountStarFunc guard matching the existing V1 method's logic.
Also added regression tests for both Iceberg V2 and V3 (Deletion Vector)
tables.
Testing:
* Added V2 regression test with SELECT 1 on table with position deletes
* Added V3 test covering count(*) optimization and the IMPALA-14993 fix
Change-Id: I8eb13aecef6d3b7460f7201dce6e03d6f3de303a
Assisted-by: Claude Opus 4.6 <[email protected]>
Reviewed-on: http://gerrit.cloudera.org:8080/24440
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Iceberg V2 count(*) optimization is incorrectly applied to queries without
> count(*), causing row loss
> -----------------------------------------------------------------------------------------------------
>
> Key: IMPALA-14993
> URL: https://issues.apache.org/jira/browse/IMPALA-14993
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.5.0
> Reporter: Dmitriy Maslov
> Assignee: Zoltán Borók-Nagy
> Priority: Major
> Labels: iceberg, impala-iceberg, impala-iceberg-active-backlog
>
> On Iceberg V2 tables that contain delete files, queries without {{count(*)}}
> in the select list (e.g. {{{}SELECT 1 FROM tbl{}}}) silently return fewer
> rows than they should.
> h3. Steps to reproduce
> {{CREATE TABLE ice1 (id INT, c1 INT)}}
> {{STORED AS ICEBERG TBLPROPERTIES ('format-version' = '2');}}
> {{INSERT INTO ice1 SELECT 1, 10;}}
> {{INSERT INTO ice1 SELECT 2, 20;}}
> {{DELETE FROM ice1 WHERE id = 1;}}
> {{SELECT 1 FROM ice1; – expected: 1 row, actual: 0 rows}}
> h3. Root cause
> {{SelectStmt.optimizePlainCountStarQueryV2()}} decides to enable the
> optimization based on a loop that _rejects_ anything that is not {{count(*)}}
> or a constant - but never checks that at least one {{count(*)}} is actually
> present. For {{SELECT 1 FROM ice1}} the loop accepts the constant and falls
> through, setting {{{}tableRef.setOptimizeCountStarForIcebergV2(true){}}}.
> h3. Proposed fix
> Implement the protection in method V2 in a similar way to method V1, by
> adding the hasCountStarFunc flag in file
> fe/src/main/java/org/apache/impala/analysis/SelectStmt.java -
> optimizePlainCountStarQueryV2() :
> {{boolean hasCountStarFunc = false;}}
> {{boolean alreadyOptimized = false;}}
> {{for (SelectListItem selectItem : getSelectList().getItems()) {}}
> {{ Expr expr = selectItem.getExpr();}}
> {{ if (expr == null) return;}}
> {{ if (expr.isConstant()) continue;}}
> {{ if (expr instanceof IcebergV2CountStarAccumulator) {}}
> {{ alreadyOptimized = true;}}
> {{ continue;}}
> {{ if (!FunctionCallExpr.isCountStarFunctionCallExpr(expr)) return;}}
> {{ hasCountStarFunc = true;}}
> {{}}}
> {{if (!hasCountStarFunc && !alreadyOptimized) return;}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
