[
https://issues.apache.org/jira/browse/YARN-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098219#comment-15098219
]
Sunil G commented on YARN-3215:
-------------------------------
Hi [~Naganarasimha Garla]
As discussed offline, I will try explaining the problem more clearly.
{{getQueueMaxResource}} considers
{{queueCapacities.getAbsoluteMaximumCapacity}} to calculate max possible
resource limit per-label. As per the approach, we try to find the difference
with this limit to actual resource used for that label (in a queue) to get the
headroom. As we consider absolute max capacity, there are chances that this
limit may reach more than 100% . (on possibility is that same label is used in
another queue).
for eg:
label X is configured in queueA.
- capacity is 30%
- max capacity is 60%
label X is configured in queueB.
- capacity is 70%
- max capcity is 80%
IN this case, its possible that we may get more than 100% for label X resource
limit. So headroom calculated per queue may be more than what is already
available. I think this is not very major problem, but its better to solved by
putting a max cap with total resource limit per label vs its used limit.
Thoughts?
> Respect labels in CapacityScheduler when computing headroom
> -----------------------------------------------------------
>
> Key: YARN-3215
> URL: https://issues.apache.org/jira/browse/YARN-3215
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: YARN-3215.v1.001.patch
>
>
> In existing CapacityScheduler, when computing headroom of an application, it
> will only consider "non-labeled" nodes of this application.
> But it is possible the application is asking for labeled resources, so
> headroom-by-label (like 5G resource available under node-label=red) is
> required to get better resource allocation and avoid deadlocks such as
> MAPREDUCE-5928.
> This JIRA could involve both API changes (such as adding a
> label-to-available-resource map in AllocateResponse) and also internal
> changes in CapacityScheduler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
