[
https://issues.apache.org/jira/browse/YARN-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355145#comment-14355145
]
patrick white commented on YARN-3215:
-------------------------------------
Hi, i am trying to reproduce the fail case as part of Label feature
verification for our usecases, and so far the headroom calculation appears to
behave correctly. Would it be possible to provide a specific fail scenario for
this issue?
There were a number of challenges getting the yarn-site and capacity properties
correctly set, we believe we have those in place now. So with both cases of
jobs running on labelled and non-labelled resources, we are seeing the task
execution staying on the correct nodes (a labelled job will only task out to
matching-labelled nodes, a non-labelled job will not task to labelled nodes)
and the headroom calc from AM logs show headroom memory dropping to 0 within 5
seconds of job start. This is observed even with small-capacity run queues for
the jobs and having 'slowstart' set to 0.
> Respect labels in CapacityScheduler when computing headroom
> -----------------------------------------------------------
>
> Key: YARN-3215
> URL: https://issues.apache.org/jira/browse/YARN-3215
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Wangda Tan
> Assignee: Wangda Tan
>
> In existing CapacityScheduler, when computing headroom of an application, it
> will only consider "non-labeled" nodes of this application.
> But it is possible the application is asking for labeled resources, so
> headroom-by-label (like 5G resource available under node-label=red) is
> required to get better resource allocation and avoid deadlocks such as
> MAPREDUCE-5928.
> This JIRA could involve both API changes (such as adding a
> label-to-available-resource map in AllocateResponse) and also internal
> changes in CapacityScheduler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
