[
https://issues.apache.org/jira/browse/IMPALA-11199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512371#comment-17512371
]
Tamas Mate commented on IMPALA-11199:
-------------------------------------
To remove this dependency, it would most likely require to move
from:
{{ ALTER TABLE <tbl> SET PARTITION SPEC(<partition-spec>)}}
to :
{{ ALTER TABLE <tbl> ADD PARTITION FIELD <spec>}}
{{ ALTER TABLE prod.db.sample DROP PARTITION FIELD <spec>}}
It seems that the Iceberg API is prepared for these types of incremental
updates
([UpdatePartitionSpec|https://github.com/apache/iceberg/blob/master/api/src/main/java/org/apache/iceberg/UpdatePartitionSpec.java#L32]
for ref) at the moment. Also, this is how Spark handles the partition
evolution
([docs|https://iceberg.apache.org/docs/latest/spark-ddl/#alter-table-sql-extensions]
for ref).
> Use UpdatePartitionSpec API to alter Iceberg partition specs
> ------------------------------------------------------------
>
> Key: IMPALA-11199
> URL: https://issues.apache.org/jira/browse/IMPALA-11199
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 4.1.0
> Reporter: Tamas Mate
> Assignee: Tamas Mate
> Priority: Major
> Labels: impala-iceberg
>
> Impala uses the iceberg TableMetadata object dugin
> [alterTableSetPartitionSpec
> |https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/IcebergCatalogOpExecutor.java#L166].
> This class is part of the core package which ideally should not be accessed
> by external applications:
> {code:java}
> BaseTable iceTable = (BaseTable)IcebergUtil.loadTable(feTable);
> TableOperations tableOp = iceTable.operations();
> TableMetadata metadata = tableOp.current();
> Schema schema = metadata.schema();
> PartitionSpec newPartSpec = IcebergUtil.createIcebergPartition(schema,
> partSpec);
> TableMetadata newMetadata = metadata.updatePartitionSpec(newPartSpec);
> Map<String, String> properties = new HashMap<>(newMetadata.properties());
> {code}
> Iceberg has an
> [UpdatePartitionSpec|https://github.com/apache/iceberg/blob/master/api/src/main/java/org/apache/iceberg/UpdatePartitionSpec.java#L32]
> api, which should be used in these situations if possible.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
