[ https://issues.apache.org/jira/browse/YARN-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509968#comment-15509968 ]
Daniel Templeton commented on YARN-5658: ---------------------------------------- Thanks for pointing that one out, [~rohithsharma]. This JIRA could be implemented using YARN-2261, but it would still be nice to have a built-in option to purge a directory. There are downstream components that need to do cleanup, like Hive and Spark. Spark could adapt their AM to use YARN-2261, but Hive couldn't. The ideal would be to allow a directory to be purged that is set through a config param. > YARN should have a hook to delete a path from HDFS when an application ends > --------------------------------------------------------------------------- > > Key: YARN-5658 > URL: https://issues.apache.org/jira/browse/YARN-5658 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager > Reporter: Daniel Templeton > Assignee: Daniel Templeton > > There are many cases when a client uploads data to HDFS and then needs to > subsequently clean it up, such as with the distributed cache. It would be > helpful if YARN would do that cleanup automatically on job completion. > The hook could be generic to an URI supported by {{FileSystem}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org