Varun Saxena commented on YARN-2902:

[~jlowe], I looked into the code.

When a container is killed during localizing state, Container cleanup is called 
which deletes the container related directory which is used for localization.
The associated LocalizerRunner Thread is also stopped. Probably at this stage 
resource should move out of DOWNLOADING state as well.
Refer to {{ResourceLocalizationService#handleCleanupContainerResources}}

Was it a PUBLIC or PRIVATE resource which was left in the DOWNLOADING state ?

> Killing a container that is localizing can orphan resources in the 
> ------------------------------------------------------------------------------------
>                 Key: YARN-2902
>                 URL: https://issues.apache.org/jira/browse/YARN-2902
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>    Affects Versions: 2.5.0
>            Reporter: Jason Lowe
>            Assignee: Varun Saxena
>             Fix For: 2.7.0
>         Attachments: YARN-2902.002.patch, YARN-2902.patch
> If a container is in the process of localizing when it is stopped/killed then 
> resources are left in the DOWNLOADING state.  If no other container comes 
> along and requests these resources they linger around with no reference 
> counts but aren't cleaned up during normal cache cleanup scans since it will 
> never delete resources in the DOWNLOADING state even if their reference count 
> is zero.

This message was sent by Atlassian JIRA

Reply via email to