Sunil G created YARN-1799:
-----------------------------
Summary: Enhance LocalDirAllocator in NM to consider
DiskMaxUtilization cutoff
Key: YARN-1799
URL: https://issues.apache.org/jira/browse/YARN-1799
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager
Affects Versions: 2.3.0
Reporter: Sunil G
LocalDirAllocator provides paths for all tasks for its local write.
This considers the good list of directories which are selected by the
HealthCheck mechamnism in LocalDirsHandlerService
getLocalPathForWrite() considers whether input demand "size" can meet the
"capacity" in "lastAccessed" directory.
If more tasks asks for path from LocalDirAllocator, then it is possible that
the allocation is done based on the current disk availability at that given
time.
But this path would have earlier given to some other tasks to write and they
may be sequentially doing writing.
It is better to check for an upper cutoff for disk availability
--
This message was sent by Atlassian JIRA
(v6.2#6252)
