[jira] [Commented] (HIVE-25894) Table migration to Iceberg doesn't remove HMS partitions
[
https://issues.apache.org/jira/browse/HIVE-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481685#comment-17481685
]
Zoltán Borók-Nagy commented on HIVE-25894:
--
Yeah, in Impala we have a check that if partition keys are empty, then there
shouldn't be any partitions:
[https://github.com/apache/impala/blob/12118664d84599f4164494058d35a882ba2dd193/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java#L685-L686]
I can change our preconditions, but if you also consider this to be a bug, then
I think it's better to fix at the Hive side.
> Table migration to Iceberg doesn't remove HMS partitions
>
>
> Key: HIVE-25894
> URL: https://issues.apache.org/jira/browse/HIVE-25894
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltán Borók-Nagy
>Priority: Major
>
> Repro:
> {code:java}
> create table ice_part_migrate (i int) partitioned by (p int) stored as
> parquet;
> insert into ice_part_migrate partition(p=1) values (1), (11), (111);
> insert into ice_part_migrate partition(p=2) values (2), (22), (222);
> ALTER TABLE ice_part_migrate SET TBLPROPERTIES
> ('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler');
> {code}
> Then looking at the HMS database:
> {code:java}
> => select "PART_NAME" from "PARTITIONS" p, "TBLS" t where
> t."TBL_ID"=p."TBL_ID" and t."TBL_NAME"='ice_part_migrate';
> PART_NAME
> ---
> p=1
> p=2
> {code}
> This is weird because Iceberg tables are supposed to be unpartitioned. It
> also breaks some precondition checks in Impala. Is there a particular reason
> to keep the partitions in HMS?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HIVE-25894) Table migration to Iceberg doesn't remove HMS partitions
[
https://issues.apache.org/jira/browse/HIVE-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481669#comment-17481669
]
Marton Bod commented on HIVE-25894:
---
[~boroknagyz] interesting, thanks for raising this! If you load the table from
HMS, it is unpartitioned:
table.getPartitionKeys() gives back an empty list. Not sure why the partitions
are not purged from the database too and whether it causes any problems.
> Table migration to Iceberg doesn't remove HMS partitions
>
>
> Key: HIVE-25894
> URL: https://issues.apache.org/jira/browse/HIVE-25894
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltán Borók-Nagy
>Priority: Major
>
> Repro:
> {code:java}
> create table ice_part_migrate (i int) partitioned by (p int) stored as
> parquet;
> insert into ice_part_migrate partition(p=1) values (1), (11), (111);
> insert into ice_part_migrate partition(p=2) values (2), (22), (222);
> ALTER TABLE ice_part_migrate SET TBLPROPERTIES
> ('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler');
> {code}
> Then looking at the HMS database:
> {code:java}
> => select "PART_NAME" from "PARTITIONS" p, "TBLS" t where
> t."TBL_ID"=p."TBL_ID" and t."TBL_NAME"='ice_part_migrate';
> PART_NAME
> ---
> p=1
> p=2
> {code}
> This is weird because Iceberg tables are supposed to be unpartitioned. It
> also breaks some precondition checks in Impala. Is there a particular reason
> to keep the partitions in HMS?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HIVE-25894) Table migration to Iceberg doesn't remove HMS partitions
[
https://issues.apache.org/jira/browse/HIVE-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481372#comment-17481372
]
Zoltán Borók-Nagy commented on HIVE-25894:
--
[~mbod] or [~lpinter] could you please take a look? Thanks.
> Table migration to Iceberg doesn't remove HMS partitions
>
>
> Key: HIVE-25894
> URL: https://issues.apache.org/jira/browse/HIVE-25894
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltán Borók-Nagy
>Priority: Major
>
> Repro:
> {code:java}
> create table ice_part_migrate (i int) partitioned by (p int) stored as
> parquet;
> insert into ice_part_migrate partition(p=1) values (1), (11), (111);
> insert into ice_part_migrate partition(p=2) values (2), (22), (222);
> ALTER TABLE ice_part_migrate SET TBLPROPERTIES
> ('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler');
> {code}
> Then looking at the HMS database:
> {code:java}
> => select "PART_NAME" from "PARTITIONS" p, "TBLS" t where
> t."TBL_ID"=p."TBL_ID" and t."TBL_NAME"='ice_part_migrate';
> PART_NAME
> ---
> p=1
> p=2
> {code}
> This is weird because Iceberg tables are supposed to be unpartitioned. It
> also breaks some precondition checks in Impala. Is there a particular reason
> to keep the partitions in HMS?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
