Wangda Tan commented on YARN-2495:

[~Naganarasimha], [~aw], let me first give you an overview about what we need 
to do to support labels in capacity scheduler, that will help you better 
understanding why we need central node label validation now. 
In existing capacity scheduler (patch of YARN-2496), we can support specify 
what labels of each queue can access (to make sure important resource can only 
be used by privileged users), and proportion of resource on label ("marketing" 
queue can access 80% of GPU resource). Now if user want to leverage change of 
capacity scheduler, user *MUST* specify 1) labels can be accessed by the queue 
and 2) proportion of resource can be accessed by a queue of each label.
Back to the central node label validation discussion, without this, we cannot 
get capacity scheduler work for now. (user cannot specify capacity for a 
unknown node-label for a queue, etc.). So I still insist to have central node 
label valication for both centralized/distribtued node label configuration at 
least for 2.6 release. This might be changed in the future, I suggest to move 
disable central node label configuration to a separated task for further 

And I've looked at patch uploaded by [~Naganarasimha], thanks for this WIP 
patch, took a quick glance at the patch, several suggestions on this patch:
- According to above comments, do not change {{CommonNodeLabelsManager}}, move 
the changes to disable central node label validation to a separated patch for 
further discussion. 
- Make this patch contains a {{NodeLabelProvider}} only and create separate 
JIRA for {{ScriptNodeLabelProvider}} and an implementation to read node label 
from yarn-site.xml for easier review.

> Allow admin specify labels in each NM (Distributed configuration)
> -----------------------------------------------------------------
>                 Key: YARN-2495
>                 URL: https://issues.apache.org/jira/browse/YARN-2495
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: YARN-2495_20141022.1.patch
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml or using script 
> suggested by [~aw])
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels

This message was sent by Atlassian JIRA

Reply via email to