[
https://issues.apache.org/jira/browse/HIVE-21471?focusedWorklogId=217815&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-217815
]
ASF GitHub Bot logged work on HIVE-21471:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Mar/19 03:40
Start Date: 25/Mar/19 03:40
Worklog Time Spent: 10m
Work Description: sankarh commented on pull request #578: HIVE-21471:
Replicating conversion of managed to external table leaks HDFS files at target.
URL: https://github.com/apache/hive/pull/578#discussion_r268480500
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
##########
@@ -400,7 +405,26 @@ public void alterTable(RawStore msdb, Warehouse wh,
String catName, String dbnam
"Unable to change partition or table. Object " + e.getMessage() + "
does not exist."
+ " Check metastore logs for detailed stack.");
} finally {
- if (!success) {
+ if (success) {
+ // Txn was committed successfully.
+ // If data location is changed in replication flow, then need to
delete the old path.
+ if (replDataLocationChanged) {
+ assert(olddb != null);
+ assert(oldt != null);
+ Path deleteOldDataLoc = new Path(oldt.getSd().getLocation());
+ boolean isAutoPurge =
"true".equalsIgnoreCase(oldt.getParameters().get("auto.purge"));
+ try {
+ wh.deleteDir(deleteOldDataLoc, true, isAutoPurge, olddb);
+ LOG.info("Deleted the old data location: {} for the table: {}",
+ deleteOldDataLoc, dbname + "." + name);
+ } catch (MetaException ex) {
+ // Eat the exception as it doesn't affect the state of existing
tables.
+ // Expect, user to manually drop this path when exception and so
logging a warning.
+ LOG.warn("Unable to delete the old data location: {} for the
table: {}",
Review comment:
I think, if old dir is deleted, then metadata update was already successful.
In this case, during replay of event in next cycle would not set the flag as
the table/partition locations were already pointing to new location under base
dir.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 217815)
Time Spent: 6.5h (was: 6h 20m)
> Replicating conversion of managed to external table leaks HDFS files at
> target.
> -------------------------------------------------------------------------------
>
> Key: HIVE-21471
> URL: https://issues.apache.org/jira/browse/HIVE-21471
> Project: Hive
> Issue Type: Bug
> Components: repl
> Affects Versions: 4.0.0
> Reporter: Sankar Hariappan
> Assignee: Sankar Hariappan
> Priority: Major
> Labels: DR, pull-request-available, replication
> Attachments: HIVE-21471.01.patch, HIVE-21471.02.patch,
> HIVE-21471.03.patch
>
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> While replicating the ALTER event to convert managed table to external table,
> the data location for the table is changed under input base directory for
> external tables replication. But, the old location remains there and would be
> leaked for ever.
> ALTER TABLE T1 SET TBLPROPERTIES('EXTERNAL'='true');
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
