[
https://issues.apache.org/jira/browse/YARN-5641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15487533#comment-15487533
]
Eric Badger commented on YARN-5641:
-----------------------------------
The problem appears to be a race between FileUtil.java:unTarUsingTar finishing
and the DeletionService deleting the localized files. If the container is
killed before the untar is finished, then it will immediately try and delete
the localized directory. Since tar is running independently of the
DeletionService, it will continue to untar even after the directory contents
have been deleted. This is why we see localized files even when the
DeletionService runs successfully.
I believe that the fix is to kill the untar process when the localize process
is killed. If we can't kill it, then we at least need to wait for it to finish
so that we can clean up the entire contents of what it untarred.
> Localizer leaves behind tarballs after container is complete
> ------------------------------------------------------------
>
> Key: YARN-5641
> URL: https://issues.apache.org/jira/browse/YARN-5641
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Eric Badger
> Assignee: Eric Badger
>
> The localizer sometimes fails to clean up extracted tarballs leaving large
> footprints that persist on the nodes indefinitely.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
