[ https://issues.apache.org/jira/browse/YARN-2791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14276107#comment-14276107 ]
Vinod Kumar Vavilapalli commented on YARN-2791: ----------------------------------------------- Okay folks, I've read the design docs on both YARN-2791 (this JIRA) and YARN-2139. This is indeed part of YARN-2139, and a direct dup of YARN-2618 and other tickets. Yes YARN-2139 is a much larger effort but it encompasses both scheduling and isolation. The important tickets of YARN-2139 already were created before this JIRA. I am going to close this as a duplicate in a day unless I see specific tasks that are not covered under YARN-2139. If there are things that are not covered indeed, I urge Swapnil Daingade, Santosh Marellaand and Yuliya Feldman to file sub-tasks under YARN-2791. As Karthik appealed before, let's have the design discussion over at YARN-2139, merging things that are only here and missing in that JIRA. Due credit will be given to all contributors to the design and implementation there. I am oblivious who contributes code, but let's work together please! > Add Disk as a resource for scheduling > ------------------------------------- > > Key: YARN-2791 > URL: https://issues.apache.org/jira/browse/YARN-2791 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler > Affects Versions: 2.5.1 > Reporter: Swapnil Daingade > Assignee: Yuliya Feldman > Attachments: DiskDriveAsResourceInYARN.pdf > > > Currently, the number of disks present on a node is not considered a factor > while scheduling containers on that node. Having large amount of memory on a > node can lead to high number of containers being launched on that node, all > of which compete for I/O bandwidth. This multiplexing of I/O across > containers can lead to slower overall progress and sub-optimal resource > utilization as containers starved for I/O bandwidth hold on to other > resources like cpu and memory. This problem can be solved by considering disk > as a resource and including it in deciding how many containers can be > concurrently run on a node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)