Naveen Gangam created HIVE-27728:
------------------------------------
Summary: Changed behavior for alter table rename partition from
legacy tables
Key: HIVE-27728
URL: https://issues.apache.org/jira/browse/HIVE-27728
Project: Hive
Issue Type: Bug
Reporter: Naveen Gangam
set hive.create.as.external.legacy=true;
CREATE TABLE default.metadata_test1(
emp_number int,
emp_name string,
city string)
PARTITIONED BY(state string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ‘,’;
INSERT INTO default.metadata_test1 PARTITION(state=‘A’) VALUES (11, ‘ABC’,
‘AA’);
INSERT INTO default.metadata_test1 PARTITION(state=‘B’) VALUES (12, ‘XYZ’,
‘BX’);
INSERT INTO default.metadata_test1 PARTITION(state=‘B’) VALUES (13, ‘UVW’,
‘BU’);
from hdfs
$ hdfs dfs -ls /warehouse/tablespace/external/hive/metadata_test1
Found 2 items
drwxr-xr-x - hive hive 0 2023-08-11 14:24
/warehouse/tablespace/external/hive/metadata_test1/state=A
drwxr-xr-x - hive hive 0 2023-08-11 14:25
/warehouse/tablespace/external/hive/metadata_test1/state=B
Now when we alter the partition.
ALTER TABLE default.metadata_test1 PARTITION (state=‘A’) RENAME TO PARTITION
(state=‘C’);
select * from default.metadata_test1;
+----------------------------+--------------------------+----------------------+-----------------------+
| metadata_test1.emp_number | metadata_test1.emp_name | metadata_test1.city |
metadata_test1.state |
+----------------------------+--------------------------+----------------------+-----------------------+
| 12 | XYZ | BX | B |
| 13 | UVW | BU | B |
| 11 | ABC | AA | C |
+----------------------------+--------------------------+----------------------+-----------------------+
show partitions default.metadata_test1;
+------------+
| partition |
+------------+
| state=B |
| state=C |
+------------+
But from HDFS
hdfs dfs -ls /warehouse/tablespace/external/hive/metadata_test1
Found 2 items
drwxr-xr-x - hive hive 0 2023-08-11 14:24
/warehouse/tablespace/external/hive/metadata_test1/state=A
drwxr-xr-x - hive hive 0 2023-08-11 14:25
/warehouse/tablespace/external/hive/metadata_test1/state=B
Hive only relocates/renames the partition locations to match the new partition
value if it is a MANAGED table. For external tables, we assume that ETL
pipelines are writing to the old location and thus will not rename the dir.
But for legacy tables, (create table) that would have created non-acid managed
tables, we convert this tables to EXTERNAL_TABLE but with auto-purge enabled.
For such tables, we should take the liberty to managed these locations as well
to match the legacy MANAGED table behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
