Craig Welch commented on YARN-2495:

Sorry if I'm jumping in late and asking redundant questions as a result, but 
I've gone through the various related jiras and the design documents (incl 
updates) ( and the patch :-) ) and I have some requirements related questions 
as a result.  Just a bit of background to make sure I understand things - it 
appears that we've settled on two different but related features here:

1. To enable node labels to be added or removed for a given node without 
validation against a centralized list of node labels (not on this patch, but 
relevant to the discussion) and
2. To enable node managers to specify their node labels based on local 
configuration and scripting (this patch is specific to that feature).

These are, strictly speaking, orthogonal, but may be used together and will 
provide something more in a 'combined feature'

A couple things about this feature (2) - I don't believe that it is necessary 
to add the node label configuration to the local configuration (yarn-site) or 
the heartbeat as such to enable configuration of labels for a node from the 
node in a decentralized fashion (e.g. a script on the node saying "put these 
labels on me").  This can already be accomplished using the admin cli from a 
script or calling a web service from the node (most likely the former, but 
either is possible...), so I don't think we need this change to support the 
script case, it's already possible to write a script to add a label to a node 
on the fly today without any changes.  To make this dynamic we would need 
feature 1, from the above, but that's not covered in this patch / is a separate 
discussion / and also does not require this change.  Also, I don't see how this 
change allows a script to dynamically configure labels unless it was changing 
the yarn-site or the like (I may have missed it, but I don't see that logic 
here) - and in any case, it would not be necessary to add this logic to support 
that sort of configuration as I pointed out.  Is this all just to support 
putting labels into the node-managers configuration file and introducing them 
that way?  Do we have a solid need for that?  It's not needed for the dynamic 
script case, which is all I've seen discussed here from a "requirements" 
perspective (putting it into the config file / adding it to the heartbeat is 
implementation, I don't see a requirement for it as such).

In a nutshell - do we need change 2 (this), or do we really just need change 1 
(eliminating validation of labels against a centralized list, at least as a 
configurable option)?

> Allow admin specify labels from each NM (Distributed configuration)
> -------------------------------------------------------------------
>                 Key: YARN-2495
>                 URL: https://issues.apache.org/jira/browse/YARN-2495
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: YARN-2495.20141023-1.patch, YARN-2495.20141024-1.patch, 
> YARN-2495.20141030-1.patch, YARN-2495.20141031-1.patch, 
> YARN-2495.20141119-1.patch, YARN-2495.20141126-1.patch, 
> YARN-2495.20141204-1.patch, YARN-2495.20141208-1.patch, 
> YARN-2495_20141022.1.patch
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml (YARN-2923) or 
> using script suggested by [~aw] (YARN-2729) )
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels

This message was sent by Atlassian JIRA

Reply via email to