[
https://issues.apache.org/jira/browse/IMPALA-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17791797#comment-17791797
]
ASF subversion and git services commented on IMPALA-12243:
----------------------------------------------------------
Commit c21e5212b73d4b51cd93e8080de69d3dee7bbfb7 in impala's branch
refs/heads/master from Peter Rozsa
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c21e5212b ]
IMPALA-12243: Add support for DROP PARTITION for Iceberg tables
This change adds support for DROP PARTITION operation for Iceberg
tables. Users can now execute 'ALTER TABLE table DROP PARTITION
(partition_filtering_expression)' statements where
'partition_filtering_expression' consists of at least one binary
predicate, IN predicate, IS NULL predicate or arbitrary logical
combination of these, where the predicate's term is a partitioned column
decorated with a partition transform, for example: 'day(column)' or
simply column.
To select partitions with mutating transforms (non-identity transforms),
the user must provide the transform in the selection, for example: if
column 'a' is partitioned as year(a), the user must filter it with
explicitly providing the transform, like (year(a) = "2012"). This is a
requirement for Iceberg's planFiles API. If the column is filtered
without the transform, the filtering of datafiles with strict projection
results in ambiguous results.
The binary predicate's operand can be basic comparison operators: =, !=,
<, >, <=, >=.
The IN and IS NULL predicates can be inverted with the NOT keyword.
Only constant literals are accepted currently for these predicates; for
example: (identity_string = "string") is accepted, (identity_string =
concat(identity_string, "another_string") is not accepted.
Tests:
- unit tests for IcebergUtil.getDateTransformValue
- analyzer tests for negative cases, simple cases
- e2e tests for every transform type, schema evolution and rollback
Change-Id: I2a768ba2966f570454687e02e4e6d67df46741f9
Reviewed-on: http://gerrit.cloudera.org:8080/20515
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add support for DROP PARTITION for Iceberg tables
> -------------------------------------------------
>
> Key: IMPALA-12243
> URL: https://issues.apache.org/jira/browse/IMPALA-12243
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Zoltán Borók-Nagy
> Assignee: Peter Rozsa
> Priority: Major
> Labels: impala-iceberg
>
> Add support for DROP PARTITION for Iceberg tables.
> Users should be able to run statements like the followings:
> * alter table table_a drop partition (country = 'SG')
> * alter table table_a drop partition (identity(country) = 'SG')
> * alter table table_a drop partition (dt < '2023-01-01 00:00:00')
> * alter table table_a drop partition (year(dt) < "2023")
> * alter table table_a drop partition (year(dt) < "2023" and month(dt) <
> "2023-06")
> * alter table table_a drop partition (bucket(5, s) < 2)
> * alter table table_a drop partition (truncate(5, s) = "trunc")
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
