[
https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169984#comment-14169984
]
Wangda Tan commented on YARN-2495:
----------------------------------
Hi [~Naganarasimha],
First I think need to take care is, distributed configuration should be a way
to input and store node labels configuration. In centralized configuration,
user can input labels and node to labels mappings by REST API and RM admin CLI,
and they will be persist to some file system like HDFS/ZK.
For distributed configuration, node labels are input by NMs (either by set
yarn-site.xml or some script), and they will be reported to RM when NM
registration. We may not need to persist any of them, but RM should know these
labels existence to do scheduling.
Another question is if we need check labels when they registering, I prefer to
pre-set them because this affects scheduling behavior. For example, the
maximum-resource and minimum-resource are setup in RM side, and RackResolver is
also run in RM side. At least, the label checking should be kept configurable
in distributed mode. -- "just ignore all the labels for that node if invalid
labels exists" might be a good way when it enabled.
For implementation, I suggest you can take a look at YARN-2494 and
YARN-2496/YARN-2500, the distributed configuration should be an extension of
NodeLabelsManager, and YARN-2496/YARN-2500 shouldn't be modified too much to
support the distributed mode in scheduling.
Thanks,
Wangda
> Allow admin specify labels in each NM (Distributed configuration)
> -----------------------------------------------------------------
>
> Key: YARN-2495
> URL: https://issues.apache.org/jira/browse/YARN-2495
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
>
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml or using script
> suggested by [~aw])
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
