Allen Wittenauer commented on YARN-796:

I agree pretty much completely with everything Sandy said, especially on the 
centralized configuration.  It actually makes configuration harder for 
heterogeneous node setups.

One caveat:

I'm also skeptical about the need for adding/removing labels dynamically. Do we 
have concrete use cases for this?

If you have the nodemanager push the labels to the RM (esp if you can do this 
via user defined script or java class...), you basically have to have dynamic 
labels for nodes.  Use cases are pretty easy to hit if you label nodes based 
upon the software stack installed. A quick example for those not following:

# User writes software that depends upon a particular version of libfoo.so.2.
# Configuration management does an install of  libfoo.so.2
# NodeManager label script picks up that it has both libfoo.so.1 and 
libfoo.so.2.  Publishes that it now has "libfoo1" and "libfoo2".  (Remember, 
this is C and not the screwed up Java universe so having two versions is 
completely legitimate)
# system can now do operations appropriate for either libfoo on that node.
# libfoo1 gets deprecated and removed from the system, again via configuration 
# label script picks up change and removes libfoo1 from label listing
# system acts appropriately and no longer does operations on node based upon 
libfoo1 label

... and all without restarting or reconfiguring anything on the Hadoop side.  
If there is any sort of manual step required in configuration the nodes short 
of the initial label script/class and other obviously user-provided bits, then 
we've failed.

> Allow for (admin) labels on nodes and resource-requests
> -------------------------------------------------------
>                 Key: YARN-796
>                 URL: https://issues.apache.org/jira/browse/YARN-796
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun C Murthy
>            Assignee: Wangda Tan
>         Attachments: LabelBasedScheduling.pdf, 
> Node-labels-Requirements-Design-doc-V1.pdf, YARN-796.patch
> It will be useful for admins to specify labels for nodes. Examples of labels 
> are OS, processor architecture etc.
> We should expose these labels and allow applications to specify labels on 
> resource-requests.
> Obviously we need to support admin operations on adding/removing node labels.

This message was sent by Atlassian JIRA

Reply via email to