[
https://issues.apache.org/jira/browse/HIVE-28727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shohei Okumiya updated HIVE-28727:
----------------------------------
Description:
We implement a logic projecting partition columns. Based on the advice of
[~dkuzmenko], we will rewrite them using Iceberg's Java APIs.
[https://github.com/apache/hive/blob/0759352ddddc793c0e717c460f0e08eb3f14c1e9/iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java#L420-L451]
was:
OPTIMIZE TABLE ... REWRITE DATA or ALTER TABLE ... COMPACT 'major' fails when
null partitions are included.
{code:java}
[ERROR] Failures:
[ERROR] TestIcebergLlapLocalCompactorCliDriver.testCliDriver:89 Client
execution failed with error code = 40000
running
alter table ice_orc COMPACT 'major' and wait
fname=iceberg_major_compaction_partition_evolution.q
See ./ql/target/tmp/log/hive.log or ./itests/qtest/target/tmp/log/hive.log, or
check ./ql/target/surefire-reports or ./itests/qtest/target/surefire-reports/
for specific test cases logs.
java.lang.RuntimeException: Error retrieving value of partition field
company_id
at
org.apache.iceberg.mr.hive.IcebergTableUtil.lambda$toPartitionData$20(IcebergTableUtil.java:399)
at java.util.Optional.orElseThrow(Optional.java:290)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.toPartitionData(IcebergTableUtil.java:398)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.lambda$null$25(IcebergTableUtil.java:483)
at org.apache.iceberg.io.CloseableIterable$7$1.next(CloseableIterable.java:202)
at org.apache.iceberg.io.FilterIterator.advance(FilterIterator.java:65)
at org.apache.iceberg.io.FilterIterator.hasNext(FilterIterator.java:49)
at java.lang.Iterable.forEach(Iterable.java:74)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.lambda$getPartitionInfo$28(IcebergTableUtil.java:492)
at java.lang.Iterable.forEach(Iterable.java:75)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.getPartitionInfo(IcebergTableUtil.java:478)
at
org.apache.iceberg.mr.hive.IcebergTableUtil.getPartitionNames(IcebergTableUtil.java:522)
at
org.apache.iceberg.mr.hive.HiveIcebergStorageHandler.getPartitions(HiveIcebergStorageHandler.java:2023)
at
org.apache.hadoop.hive.ql.metadata.HiveStorageHandler.getPartitions(HiveStorageHandler.java:888)
at org.apache.hadoop.hive.ql.metadata.Hive.getPartitions(Hive.java:4150)
at
org.apache.hadoop.hive.ql.ddl.table.storage.compact.AlterTableCompactOperation.getPartitions(AlterTableCompactOperation.java:175)
at
org.apache.hadoop.hive.ql.ddl.table.storage.compact.AlterTableCompactOperation.execute(AlterTableCompactOperation.java:88)
at org.apache.hadoop.hive.ql.ddl.DDLTask.execute(DDLTask.java:84) {code}
> Iceberg: Refactor IcebergTableUtil.toPartitionData
> --------------------------------------------------
>
> Key: HIVE-28727
> URL: https://issues.apache.org/jira/browse/HIVE-28727
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
> Reporter: Shohei Okumiya
> Assignee: Shohei Okumiya
> Priority: Major
> Labels: pull-request-available
>
> We implement a logic projecting partition columns. Based on the advice of
> [~dkuzmenko], we will rewrite them using Iceberg's Java APIs.
> [https://github.com/apache/hive/blob/0759352ddddc793c0e717c460f0e08eb3f14c1e9/iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java#L420-L451]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
