Dian Fu commented on YARN-3964:

Hi [~Naganarasimha], [~sunilg],
Thanks a lot for your review. Updated the patch according to your comments with 
the following exceptions:
If updateInterval is disabled then only on register of nodes the labels are 
fetched, is that fine or is it better to fetch once and not run timer ?
You mean not only fetching the labels on register, but also fetching the labels 
once for all the nodes when all nodes are registered? If it's so, I think 
this's unnecessary and difficult to implement. Firstly, the labels of all the 
registered nodes have been fetched once on register. Secondly, it's difficult 
to determine when to fetch the labels for all the nodes as some nodes may still 
not registered. Thoughts?
createNodeLabelsMappingProvider throws exception from serviceInit if 
NodeLabelsMappingProvider class is not configured.
I would agree with [~Naganarasimha] on this point. By default, {{centralized}} 
node label configuration type is enabled. If user explicitly set the node label 
configuration type to {{delegated-centralized}}, we can assume that he knows 
what does it mean for {{delegated-centralized}} and he knows he should provide 
a {{NodeLabelsMappingProvider}}. On the other hand, providing a 
{{SimpleNodeLabelsMappingProvider}} and make it as default would also be 
acceptable for me. If we decide to do this, then we should log the following 
with INFO to let user know which {{NodeLabelsMappingProvider}} is being used. 
{{LOG.debug("Node labels mapping provider class is : "
157               + nodeLabelsMappingProvider.getClass().toString());}} 

> Support NodeLabelsProvider at Resource Manager side
> ---------------------------------------------------
>                 Key: YARN-3964
>                 URL: https://issues.apache.org/jira/browse/YARN-3964
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Dian Fu
>            Assignee: Dian Fu
>         Attachments: YARN-3964 design doc.pdf, YARN-3964.002.patch, 
> YARN-3964.003.patch, YARN-3964.004.patch, YARN-3964.005.patch, 
> YARN-3964.006.patch, YARN-3964.007.patch, YARN-3964.007.patch, 
> YARN-3964.008.patch, YARN-3964.009.patch, YARN-3964.1.patch
> Currently, CLI/REST API is provided in Resource Manager to allow users to 
> specify labels for nodes. For labels which may change over time, users will 
> have to start a cron job to update the labels. This has the following 
> limitations:
> - The cron job needs to be run in the YARN admin user.
> - This makes it a little complicate to maintain as users will have to make 
> sure this service/daemon is alive.
> Adding a Node Labels Provider in Resource Manager will provide user more 
> flexibility.

This message was sent by Atlassian JIRA

Reply via email to