Jason Lowe commented on YARN-2473:

No worries, Varun, we all missed it. ;-)

We may need to track full disks separately from "bad" disks so we can know 
whether or not it's OK to try to delete a container directory from a particular 
disk that isn't a known "good" disk.  I'm hesitant to have the NM try to remove 
container directories even from "bad" disks since touching them can cause a 
very long pause for the thread that did it.

> YARN never cleans up container directories from a full disk
> -----------------------------------------------------------
>                 Key: YARN-2473
>                 URL: https://issues.apache.org/jira/browse/YARN-2473
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.4.0
>            Reporter: Jason Lowe
>            Priority: Blocker
> After YARN-1781 when a container ends up filling a local disk the nodemanager 
> will mark it as a bad disk and remove it from the list of good local dirs.  
> When the container eventually completes the files that filled the disk will 
> not be removed because the NM thinks the directory is bad.

This message was sent by Atlassian JIRA

Reply via email to