[
https://issues.apache.org/jira/browse/YARN-8703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598533#comment-16598533
]
lujie commented on YARN-8703:
-----------------------------
[~jlowe]
Thanks for your explain, I have added sleep and kill job command in the
constructor of ResourceLocalizedEvent, I also add a log statement to show the
suspicious file path! By doing this, i have triggered the warning log
statement to execute. The log file shows that the suspicious file can be
deleted when the application finished!
> Localized resource may leak on disk if container is killed while localizing
> ---------------------------------------------------------------------------
>
> Key: YARN-8703
> URL: https://issues.apache.org/jira/browse/YARN-8703
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Reporter: Jason Lowe
> Priority: Major
>
> If a container is killed while localizing then it releases all of its
> resources. If the resource count goes to zero and it is in the DOWNLOADING
> state then the resource bookkeeping is removed in the resource tracker.
> Shortly afterwards the localizer could heartbeat in and report the successful
> localization of the resource that was just removed. When the
> LocalResourcesTrackerImpl receives the LOCALIZED event but does not find the
> corresponding LocalResource for the event then it simply logs a "localized
> without a location" warning. At that point I think the localized resource
> has been leaked on the disk since the NM has removed bookkeeping for the
> resource without removing it on disk.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
