[
https://issues.apache.org/jira/browse/YARN-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124824#comment-15124824
]
He Tianyi commented on YARN-2139:
---------------------------------
Recently introduced SSD in my cluster for MapReduce shuffle.
Then there is one issue, if map output gets too large, it cannot be placed on
SSD. We have to implement a custom strategy (called SSDFirst) to make best
effort to use SSD, but fallbacks to HDD when available space of SSD gets tight.
This worked in most cases, but it is only a local optimum. To achieve global
optimum, scheduler must be aware and management these resources.
> [Umbrella] Support for Disk as a Resource in YARN
> --------------------------------------------------
>
> Key: YARN-2139
> URL: https://issues.apache.org/jira/browse/YARN-2139
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Wei Yan
> Attachments: Disk_IO_Isolation_Scheduling_3.pdf,
> Disk_IO_Scheduling_Design_1.pdf, Disk_IO_Scheduling_Design_2.pdf,
> YARN-2139-prototype-2.patch, YARN-2139-prototype.patch
>
>
> YARN should consider disk as another resource for (1) scheduling tasks on
> nodes, (2) isolation at runtime, (3) spindle locality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
