[
https://issues.apache.org/jira/browse/YARN-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509968#comment-15509968
]
Daniel Templeton commented on YARN-5658:
----------------------------------------
Thanks for pointing that one out, [~rohithsharma]. This JIRA could be
implemented using YARN-2261, but it would still be nice to have a built-in
option to purge a directory. There are downstream components that need to do
cleanup, like Hive and Spark. Spark could adapt their AM to use YARN-2261, but
Hive couldn't. The ideal would be to allow a directory to be purged that is
set through a config param.
> YARN should have a hook to delete a path from HDFS when an application ends
> ---------------------------------------------------------------------------
>
> Key: YARN-5658
> URL: https://issues.apache.org/jira/browse/YARN-5658
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
> Reporter: Daniel Templeton
> Assignee: Daniel Templeton
>
> There are many cases when a client uploads data to HDFS and then needs to
> subsequently clean it up, such as with the distributed cache. It would be
> helpful if YARN would do that cleanup automatically on job completion.
> The hook could be generic to an URI supported by {{FileSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
