Wangda Tan commented on YARN-2495:

1) Please remove all script related configurations in YarnConfiguration

2) It seems NM_LABELS_FETCH_INTERVAL_MS not been used in the patch, did you 
forget to do that?

3) Regarding ResourceTrackerProtocol, I think NodeHeartbeatRequest should only 
report labels when labels changed. So there're 3 possible values of node labels 
in NodeHeartbeatRequest:
   1. null: labels not changed
   2. empty array: no label on this node
   3. non-empty array: labels on this node
   And RegisterNodeManagerRequest should report label every time registering.

4) NodeLabelsFetcherService:
4.1 Why this class extends from CompositeService? Did you want to add more 
component to it? If not, AbstractService should be enough. If the purpose of 
the NodeLabelsFetcherService is only create a NodeLabelsProvider, and the 
NodeLabelsProvider will take care of periodically read configuration from 
yarn-site.xml. I suggest to rename NodeLabelsFetcherService to 
NodeLabelsProviderFactory, and not extends from any Service, because the 
NodeLabelsProvider should be a Service. Rename NodeLabelsProvider to 
NodeLabelsProviderService if your purpose is as what I mentioned.

4.2 This should be located in 
org.apache.hadoop.yarn.server.nodemanager.nodelabels, and rename 
"org.apache.hadoop.yarn.server.nodemanager.nodelabel" to 
"org.apache.hadoop.yarn.server.nodemanager.nodelabels" in other class definition

5) ConfigurationNodeLabelsProvider
This should be an independent class and located in a individual file

6) More implementation suggestions:
Since we need central node labels configuration, I suggest to leverage what we 
already have in RM admin CLI directly -- user can use RM admin CLI add/remove 
node labels. We can disable this when we're ready to do non-central node label 
And there should be an option to tell if distributed node label configuration 
is used. If it's distributed, AdminService should disable admin change labels 
on nodes via RM admin CLI. I suggest to do this in a separated JIRA.

> Allow admin specify labels in each NM (Distributed configuration)
> -----------------------------------------------------------------
>                 Key: YARN-2495
>                 URL: https://issues.apache.org/jira/browse/YARN-2495
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: YARN-2495.20141023-1.patch, YARN-2495_20141022.1.patch
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml or using script 
> suggested by [~aw])
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels

This message was sent by Atlassian JIRA

Reply via email to