Steve Carlin has posted comments on this change. ( http://gerrit.cloudera.org:8080/24311 )
Change subject: IMPALA-14999: Calcite planner: support Iceberg tables (part 2) ...................................................................... Patch Set 18: (5 comments) http://gerrit.cloudera.org:8080/#/c/24311/16//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/24311/16//COMMIT_MSG@9 PS16, Line 9: The main changes enabling Iceberg tables to be processed by the Calcite planner : can be found in: : - CalciteDb: The table fetched from metadata now allows FeIcebergTable objects. : This will instantiate a CalciteIcebergTable object. The CalciteIcebergTable : object isn't explicitly needed for supporting Iceberg tables, but it does : contain methods needed for count star optimization which will be explained below. : - ImpalaHdfsScanRel: When an Iceberg table is found, it needs to instantiate : the physical nodes via the IcebergScanPlanner. : : All the other changes in this commit are meant to deal with either specific : issues within Calcite that needed to change to handle edge cases or performance : optimizations for Iceberg. These changes include: : : The biggest change in terms of code complexity is to handle the count star : optimization for Iceberg. Mo > Don't forget to remove these lines. Done http://gerrit.cloudera.org:8080/#/c/24311/16/java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/IcebergCountStarOptimizer.java File java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/IcebergCountStarOptimizer.java: http://gerrit.cloudera.org:8080/#/c/24311/16/java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/IcebergCountStarOptimizer.java@75 PS16, Line 75: be done to handle : * the delete files. The agg/tablescan hierarchy remains as is because of the extra : * sum that is needed, but a Project on top adds the precalculated count to the : * count needed to handle the delete files. The output in the project RelNode for : * the count(*) will be "InputRef($1) + <precalculated count(*) constant>". : * > Outdated due to L115. Done http://gerrit.cloudera.org:8080/#/c/24311/16/java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/IcebergCountStarOptimizer.java@112 PS16, Line 112: try { : > nit: extra empty line Done http://gerrit.cloudera.org:8080/#/c/24311/16/java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/IcebergCountStarOptimizer.java@147 PS16, Line 147: or the count(*), we add the input ref to the count literal constant from the : // v2 count. : projects.add(rexBuilder.makeCall(ImpalaCustomOperatorTable.PLUS, aggInput, : countLiteral)); : } > Dead code? Done http://gerrit.cloudera.org:8080/#/c/24311/16/testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test File testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test: http://gerrit.cloudera.org:8080/#/c/24311/16/testdata/workloads/functional-query/queries/QueryTest/iceberg-plain-count-star-optimization.test@11 PS16, Line 11: count(*) > Can we add test for 'count(*) + 1000' as well? Currently it is not optimize Added it both here and the v2 version -- To view, visit http://gerrit.cloudera.org:8080/24311 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1854012b1caac63ced292b338d40074db950b42d Gerrit-Change-Number: 24311 Gerrit-PatchSet: 18 Gerrit-Owner: Steve Carlin <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Thu, 02 Jul 2026 20:54:01 +0000 Gerrit-HasComments: Yes
