Daniel Templeton commented on YARN-5658:

Thanks for pointing that one out, [~rohithsharma].  This JIRA could be 
implemented using YARN-2261, but it would still be nice to have a built-in 
option to purge a directory.  There are downstream components that need to do 
cleanup, like Hive and Spark.  Spark could adapt their AM to use YARN-2261, but 
Hive couldn't.  The ideal would be to allow a directory to be purged that is 
set through a config param.

> YARN should have a hook to delete a path from HDFS when an application ends
> ---------------------------------------------------------------------------
>                 Key: YARN-5658
>                 URL: https://issues.apache.org/jira/browse/YARN-5658
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
> There are many cases when a client uploads data to HDFS and then needs to 
> subsequently clean it up, such as with the distributed cache.  It would be 
> helpful if YARN would do that cleanup automatically on job completion.
> The hook could be generic to an URI supported by {{FileSystem}}.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to