[
https://issues.apache.org/jira/browse/IMPALA-14675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18058257#comment-18058257
]
ASF subversion and git services commented on IMPALA-14675:
----------------------------------------------------------
Commit 1a814f5f01f1f401517f252aeefe906a2ef1c87a in impala's branch
refs/heads/master from Arnab Karmakar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=1a814f5f0 ]
IMPALA-14555: Add Iceberg support for SHOW PARTITIONS WHERE
This patch adds WHERE clause support for SHOW PARTITIONS on Iceberg tables,
extending the functionality previously added for HDFS tables in IMPALA-14065.
Iceberg uses manifest-based file/partition filtering instead of filesystem
traversal. This initial implementation supports simple predicates
(comparisons, IN, BETWEEN, IS NULL, AND/OR) but not functions, subqueries, or
analytics.
For complex queries with functions, use Iceberg metadata tables.
The following is not supported in SHOW PARTITIONS:
SHOW PARTITIONS functional_parquet.iceberg_partitioned
WHERE upper(action) = 'CLICK';
Use the metadata table instead:
SELECT `partition`
FROM functional_parquet.iceberg_partitioned.`partitions`
WHERE upper(`partition`.action) = 'CLICK';
This change also adds DROP PARTITION coverage for BETWEEN predicates on
Iceberg partition columns in iceberg-drop-partition.test.
Testing:
- Unit tests in AnalyzeDDLTest for valid predicates and error cases
- Positive test cases for simple predicates, BETWEEN, IN, IS NULL, AND, OR,
NOT, and boolean literals
- Negative test cases that reject unsupported functions with clear errors
- DROP PARTITION tests with BETWEEN predicates in
testdata/workloads/functional-query/queries/QueryTest/iceberg-drop-partition.test
TODO:
IMPALA-14675: Support complex predicates (SQL functions, non-deterministic
functions, aggregates, analytics, subqueries) in SHOW PARTITIONS WHERE for
Iceberg tables by evaluating them per-partition using
FeSupport.EvalPredicateBatch(), similar to HDFS tables.
Generated-by: Cursor AI (claude-4.5-sonnet)
Change-Id: I4c0ee4d171ae939770725d89dc504e13f82a7688
Reviewed-on: http://gerrit.cloudera.org:8080/23800
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Support complex predicates in SHOW PARTITIONS WHERE for Iceberg tables
> ----------------------------------------------------------------------
>
> Key: IMPALA-14675
> URL: https://issues.apache.org/jira/browse/IMPALA-14675
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend, Frontend
> Reporter: Arnab Karmakar
> Priority: Major
>
> IMPALA-14555 added basic WHERE clause support for Iceberg SHOW PARTITIONS,
> but only supports simple predicates that can be converted to Iceberg
> expressions (e.g., column comparisons, BETWEEN, IN, IS NULL, AND, OR, NOT).
> Complex predicates are currently not supported:
> - SQL functions (e.g., SHOW PARTITIONS WHERE upper(action) = 'CLICK')
> - Non-deterministic functions (e.g., SHOW PARTITIONS WHERE rand() < 0.5)
> - Aggregate functions
> - Analytic expressions
> For such queries, users must currently use Iceberg metadata tables:
> {code:java}
> SELECT * FROM table.partitions WHERE upper(action) = 'CLICK'{code}
> This follow-up work should add support for complex predicates by evaluating
> them per-partition using FeSupport.EvalPredicateBatch(), similar to how HDFS
> tables currently work (see HdfsPartitionPruner.prunePartitions with
> evalAllFuncs=true).
> This would provide feature parity between HDFS and Iceberg tables for SHOW
> PARTITIONS WHERE.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
