[
https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068041#comment-14068041
]
Jian Fang commented on YARN-796:
--------------------------------
As Sandy pointed out, seems the scope becomes bigger and bigger. Take our use
case as an example, we initial only need to restrict Application masters not be
assigned to some nodes such as spot instances in EC2. In our design, we only
added the following parameters
yarn.label.enabled
yarn.nodemanager.labels
yarn.app.mapreduce.am.labels
to yarn-site.xml and then modified hadoop code. This function works now. With
the current proposal, I wonder how long it may take to finish.
I also doubt about the assumption for admin to configure labels for a cluster.
Usually a cluster comes with hundreds or thousands of nodes, how possible for
the admin to manually configure the labels? This type of work can be easily
automated by some script or a java process running on each node to write the
labels such as OS, processor, and other parameters to yarn-site.xml before the
cluster is started. This is especially true for clusters in a cloud because
everything is automated there. The admin UI could only be used in some special
cases that require human intervention.
One use case for dynamic labeling is that we can put a label to a node when we
try to shrink a cluster so that hadoop will not assign tasks to that node any
more to give that node some grace time to be decommissioned. This is most
likely to be implemented by a restful API call from a process that chooses a
node to remove based on cluster metrics of the cluster.
> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>
> Key: YARN-796
> URL: https://issues.apache.org/jira/browse/YARN-796
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Arun C Murthy
> Assignee: Wangda Tan
> Attachments: LabelBasedScheduling.pdf,
> Node-labels-Requirements-Design-doc-V1.pdf, YARN-796.patch
>
>
> It will be useful for admins to specify labels for nodes. Examples of labels
> are OS, processor architecture etc.
> We should expose these labels and allow applications to specify labels on
> resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
