[
https://issues.apache.org/jira/browse/YARN-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110625#comment-15110625
]
Sunil G commented on YARN-3215:
-------------------------------
Thanks [~Naganarasimha Garla] for sharing the patch.
{{headroom}} will be mostly less than {{clusterFreePartitionResource}}, and
this new min helps to ensure that we do not over-promise. Approach and patch
looks fine for me.
As discussed offline, One general improvement can be like,
{code}
Resource queueCapacity =
Resources.multiplyAndNormalizeUp(resourceCalculator,
labelManager.getResourceByLabel(nodePartition, clusterResource),
queueCapacities.getAbsoluteCapacity(nodePartition),
minimumAllocation);
Resource queueMaxCapacity =
Resources.multiplyAndNormalizeUp(resourceCalculator,
labelManager.getResourceByLabel(nodePartition, clusterResource),
queueCapacities.getAbsoluteMaximumCapacity(nodePartition),
minimumAllocation);
{code}
If we can have this values pre-computed, it will be much better. This can be
changed only when node is added/removed etc. In all other cases, we can just
use it via a read-only interface. Can we have this inside QueueCapacities.
Thoughts?
> Respect labels in CapacityScheduler when computing headroom
> -----------------------------------------------------------
>
> Key: YARN-3215
> URL: https://issues.apache.org/jira/browse/YARN-3215
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: YARN-3215.v1.001.patch, YARN-3215.v2.001.patch,
> YARN-3215.v2.002.patch
>
>
> In existing CapacityScheduler, when computing headroom of an application, it
> will only consider "non-labeled" nodes of this application.
> But it is possible the application is asking for labeled resources, so
> headroom-by-label (like 5G resource available under node-label=red) is
> required to get better resource allocation and avoid deadlocks such as
> MAPREDUCE-5928.
> This JIRA could involve both API changes (such as adding a
> label-to-available-resource map in AllocateResponse) and also internal
> changes in CapacityScheduler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
