Hello Quanlong Huang, Riza Suminto, Zoltan Borok-Nagy, Peter Rozsa, Daniel
Vanko, Mihaly Szjatinya, Michael Smith, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23800
to look at the new patch set (#10).
Change subject: IMPALA-14555: Add Iceberg support for SHOW PARTITIONS WHERE
......................................................................
IMPALA-14555: Add Iceberg support for SHOW PARTITIONS WHERE
This patch adds WHERE clause support for SHOW PARTITIONS on Iceberg tables,
extending the functionality previously added for HDFS tables in IMPALA-14065.
Iceberg uses manifest-based file/partition filtering instead of filesystem
traversal. This initial implementation supports simple predicates
(comparisons, IN, BETWEEN, IS NULL, AND/OR) but not functions, subqueries, or
analytics.
For complex queries with functions, use Iceberg metadata tables.
The following is not supported in SHOW PARTITIONS:
SHOW PARTITIONS functional_parquet.iceberg_partitioned
WHERE upper(action) = 'CLICK';
Use the metadata table instead:
SELECT `partition`
FROM functional_parquet.iceberg_partitioned.`partitions`
WHERE upper(`partition`.action) = 'CLICK';
This change also adds DROP PARTITION coverage for BETWEEN predicates on
Iceberg partition columns in iceberg-drop-partition.test.
Testing:
- Unit tests in AnalyzeDDLTest for valid predicates and error cases
- Positive test cases for simple predicates, BETWEEN, IN, IS NULL, AND, OR,
NOT, and boolean literals
- Negative test cases that reject unsupported functions with clear errors
- DROP PARTITION tests with BETWEEN predicates in
testdata/workloads/functional-query/queries/QueryTest/iceberg-drop-partition.test
TODO:
IMPALA-14675: Support complex predicates (SQL functions, non-deterministic
functions, aggregates, analytics, subqueries) in SHOW PARTITIONS WHERE for
Iceberg tables by evaluating them per-partition using
FeSupport.EvalPredicateBatch(), similar to HDFS tables.
Generated-by: Cursor AI (claude-4.5-sonnet)
Change-Id: I4c0ee4d171ae939770725d89dc504e13f82a7688
---
M common/thrift/Frontend.thrift
M docs/topics/impala_iceberg.xml
M
fe/src/main/java/org/apache/impala/analysis/IcebergPartitionExpressionRewriter.java
M fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/common/IcebergPredicateConverter.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeDDLTest.java
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-drop-partition.test
M testdata/workloads/functional-query/queries/QueryTest/show-stats.test
11 files changed, 606 insertions(+), 57 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/00/23800/10
--
To view, visit http://gerrit.cloudera.org:8080/23800
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4c0ee4d171ae939770725d89dc504e13f82a7688
Gerrit-Change-Number: 23800
Gerrit-PatchSet: 10
Gerrit-Owner: Arnab Karmakar <[email protected]>
Gerrit-Reviewer: Arnab Karmakar <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Daniel Vanko <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Mihaly Szjatinya <[email protected]>
Gerrit-Reviewer: Peter Rozsa <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
