[
https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14179673#comment-14179673
]
Wangda Tan commented on YARN-2495:
----------------------------------
[~Naganarasimha], [~aw], let me first give you an overview about what we need
to do to support labels in capacity scheduler, that will help you better
understanding why we need central node label validation now.
In existing capacity scheduler (patch of YARN-2496), we can support specify
what labels of each queue can access (to make sure important resource can only
be used by privileged users), and proportion of resource on label ("marketing"
queue can access 80% of GPU resource). Now if user want to leverage change of
capacity scheduler, user *MUST* specify 1) labels can be accessed by the queue
and 2) proportion of resource can be accessed by a queue of each label.
Back to the central node label validation discussion, without this, we cannot
get capacity scheduler work for now. (user cannot specify capacity for a
unknown node-label for a queue, etc.). So I still insist to have central node
label valication for both centralized/distribtued node label configuration at
least for 2.6 release. This might be changed in the future, I suggest to move
disable central node label configuration to a separated task for further
discussions.
And I've looked at patch uploaded by [~Naganarasimha], thanks for this WIP
patch, took a quick glance at the patch, several suggestions on this patch:
- According to above comments, do not change {{CommonNodeLabelsManager}}, move
the changes to disable central node label validation to a separated patch for
further discussion.
- Make this patch contains a {{NodeLabelProvider}} only and create separate
JIRA for {{ScriptNodeLabelProvider}} and an implementation to read node label
from yarn-site.xml for easier review.
> Allow admin specify labels in each NM (Distributed configuration)
> -----------------------------------------------------------------
>
> Key: YARN-2495
> URL: https://issues.apache.org/jira/browse/YARN-2495
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: YARN-2495_20141022.1.patch
>
>
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml or using script
> suggested by [~aw])
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
