[
https://issues.apache.org/jira/browse/YARN-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095620#comment-15095620
]
Wangda Tan commented on YARN-3215:
----------------------------------
[~Naganarasimha],
Thanks for reply, I think the optimization (i.e consider the labels only for
those resource requests with container number >0 and hostName = *) may be not
required. For example, a app could ask for resources in waves, at first want
100 containers, and after a while, ask 100 more containers. If scheduler
returns headroom only considering partitions have "active" requests,
application may think there's no room for the 100 more containers.
Another reason is, existing applications will ask less partitions. For example
MR needs at most two different partitions for tasks (one for mappers, and one
for reducers), which could less than number of different priorities.
I would suggest to consider unique partition only in this JIRA, this could be
optimized in the future when we have real life requirements.
> Respect labels in CapacityScheduler when computing headroom
> -----------------------------------------------------------
>
> Key: YARN-3215
> URL: https://issues.apache.org/jira/browse/YARN-3215
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: YARN-3215.v1.001.patch
>
>
> In existing CapacityScheduler, when computing headroom of an application, it
> will only consider "non-labeled" nodes of this application.
> But it is possible the application is asking for labeled resources, so
> headroom-by-label (like 5G resource available under node-label=red) is
> required to get better resource allocation and avoid deadlocks such as
> MAPREDUCE-5928.
> This JIRA could involve both API changes (such as adding a
> label-to-available-resource map in AllocateResponse) and also internal
> changes in CapacityScheduler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
