[ https://issues.apache.org/jira/browse/YARN-326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13643316#comment-13643316 ]
Sandy Ryza commented on YARN-326: --------------------------------- Just chatted with Karthik offline about this, and I think his approach his approach makes sense as a good hybrid. To state it a little more verbosely, a non-fractional minimum share can be configured for each resource. A queue is considered under its minimum share if the dominant resource in its current allocation with respect to the cluster capacity is below its minimum share for that resource. For a single resource, this reduces to the existing way things work. To achieve the kind of fraction of dominant resource that Andrew speaks about, one can simply configure shares for different resources on a queue at the same fractions of the cluster capacity. And if we want to emphasize the geology department's minimum share towards CPU, we can do that as well. A concrete example would be: queue A's minimum resource vector is 10 CPU, 3 GB. The cluster capacity is 15 CPU, 15 GB. If queue A already has allocated <4 CPU, 2 GB>, <1 CPU, 2 GB>, or <9 CPU, 8 GB>, it will be considered under its minimum share and get a container before any queue that is not. If queue A has <3 CPU, 4 GB> or <11 CPU, 8 GB>, it will not. While configuring shares as fractions of the cluster size would make things more resilient to changing clusters, I think that in order to maintain compatibility, we would have to allow both, and this is probably work for a different JIRA. Is there anything we're missing here? > Add multi-resource scheduling to the fair scheduler > --------------------------------------------------- > > Key: YARN-326 > URL: https://issues.apache.org/jira/browse/YARN-326 > Project: Hadoop YARN > Issue Type: New Feature > Components: scheduler > Affects Versions: 2.0.2-alpha > Reporter: Sandy Ryza > Assignee: Sandy Ryza > Attachments: FairSchedulerDRFDesignDoc.pdf, YARN-326.patch, > YARN-326.patch > > > With YARN-2 in, the capacity scheduler has the ability to schedule based on > multiple resources, using dominant resource fairness. The fair scheduler > should be able to do multiple resource scheduling as well, also using > dominant resource fairness. > More details to come on how the corner cases with fair scheduler configs such > as min and max resources will be handled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira