Song Jun created SPARK-19359:
--------------------------------
Summary: partition path created by Hive should be deleted after
rename a partition with upper-case
Key: SPARK-19359
URL: https://issues.apache.org/jira/browse/SPARK-19359
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: Song Jun
Priority: Minor
Hive metastore is not case preserving and keep partition columns with lower
case names.
If SparkSQL create a table with upper-case partion name use
HiveExternalCatalog, when we rename partition, it first call the HiveClient to
renamePartition, which will create a new lower case partition path, then
SparkSql rename the lower case path to the upper-case.
while if the renamed partition contains more than one depth partition ,e.g.
A=1/B=2, hive renamePartition change to a=1/b=2, then SparkSql rename it to
A=1/B=2, but the a=1 still exists in the filesystem, we should also delete it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
