Gang Tim Liu created HIVE-3251:
----------------------------------
Summary: Hive doesn't remove scrach directories while killing
running MR job
Key: HIVE-3251
URL: https://issues.apache.org/jira/browse/HIVE-3251
Project: Hive
Issue Type: Bug
Components: Server Infrastructure
Reporter: Gang Tim Liu
Assignee: Gang Tim Liu
While killing running MR job, hive doesn't clean up scratch directory
(mapred.cache.files). So that, afterwards, scratch directory is left there in
hdfs. HDFS name node doesn't know it and try to do lease recovery. while such
instances happen more, it will eventually crash namenode.
The fix is to leverage hdfs clean up functionality. While creating scratch
dirs, hive registers it to hdfs cleanup hook. While killing happens, hdfs will
clean them up.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
