Jian Fang commented on YARN-796:

I like to add a use case to this JIRA. 

In a cloud environment, hadoop could run in heterogeneous groups of 
instances.Take Amazon EMR as an example, usually an EMR hadoop cluster runs in 
master, core, and task groups, where the task group could be spot instances 
that can go away at any time. As a result, we like to have a tag capability on 
each node. That is to say, when a node manager starts up, it will load the tags 
from the configuration file. Then, the resource manager could refine the 
scheduling results based on the tags. 

One good example is that we don't want an application master to be assigned to 
any spot instance in a task group because that instance could be taken away by 
EC2 at any time. 

If hadoop resource could support a tag capability, then we could extend the 
current scheduling algorithm to add constraints to not assign the application 
master to a task node.

We don't really need any admin capability for the tags (but still good to have) 
since the tags are static and can be specified in a configuration file, for 
example yarn-site.xml.

> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>                 Key: YARN-796
>                 URL: https://issues.apache.org/jira/browse/YARN-796
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Wangda Tan
>         Attachments: YARN-796.patch
> It will be useful for admins to specify labels for nodes. Examples of labels 
> are OS, processor architecture etc.
> We should expose these labels and allow applications to specify labels on 
> resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.

This message was sent by Atlassian JIRA

Reply via email to