[
https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14382299#comment-14382299
]
Jian Fang commented on YARN-2495:
---------------------------------
In a cloud environment such as Amazon EMR, a hadoop cluster is launched as a
service by a single command line. There is no admin at all and everything is
automated. The lables are basically of two types, one is static. For example,
the nature of an EC2 instance such as spot or on-demand. The other is dynamic.
For example, the cluster controller process can set an instance to be a
candidate to be terminated in the case of graceful shrink so that resource
manager will not assign new tasks to it.
Most likely, the labels specified from each NM are static and are provided by a
cluster controller process to write into yarn-site.xml based on EC2 metadata
available on each EC2 instance. As a result, at least you should defined a
static lablel provider (plus a dynamic lable provider? not sure) so that these
lables are only sent to resource manager at NM registeration time. There is no
point to add the static lables to each heartbeat.
I think the idea of central and distributed label configurations are not ideal
to use in a cloud environment. Usually we have a mix of static lables from each
node and dynamic labels that are specified against the resource manager
directly. Static and dynamic lable concepts are more appopriate at least for
Amazon EMR.
> Allow admin specify labels from each NM (Distributed configuration)
> -------------------------------------------------------------------
>
> Key: YARN-2495
> URL: https://issues.apache.org/jira/browse/YARN-2495
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: YARN-2495.20141023-1.patch, YARN-2495.20141024-1.patch,
> YARN-2495.20141030-1.patch, YARN-2495.20141031-1.patch,
> YARN-2495.20141119-1.patch, YARN-2495.20141126-1.patch,
> YARN-2495.20141204-1.patch, YARN-2495.20141208-1.patch,
> YARN-2495.20150305-1.patch, YARN-2495.20150309-1.patch,
> YARN-2495.20150318-1.patch, YARN-2495.20150320-1.patch,
> YARN-2495.20150321-1.patch, YARN-2495.20150324-1.patch,
> YARN-2495_20141022.1.patch
>
>
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml (YARN-2923) or
> using script suggested by [~aw] (YARN-2729) )
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
