Dian Fu commented on YARN-3964:

Hi [~leftnoteasy],
Thanks a lot for your review and feedback.
According to the design doc, configuration of centralized provider also needs 
to consider 1) frequency of invoking fetch script. 
User just need to configure it via configuration and a default value can be 
provided if user doesn't configure. 
2) permission of script. 
Sorry, I'm not sure what's your point? You mean the execution permission? I 
don't think that would be a problem. Actually the provider don't have to be 
script based.
3) a customized provider
The implementation of a customized provider will be very simple. User just need 
to implement interface {{NodeLabelsFetcher}}.
4) configuration RM classpath to make use of it. 
Yes, that's true.
Configuration/executing provider script also needs YARN's admin permission.
The provider will be part of Resource Manager, we don't need to start up a 
separate process to execute it. 

To clearly express the thoughts, I have uploaded an initial patch. 
Thanks again for your comments.

> Support NodeLabelsProvider at Resource Manager side
> ---------------------------------------------------
>                 Key: YARN-3964
>                 URL: https://issues.apache.org/jira/browse/YARN-3964
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Dian Fu
>            Assignee: Dian Fu
>         Attachments: YARN-3964 design doc.pdf, YARN-3964.1.patch
> Currently, CLI/REST API is provided in Resource Manager to allow users to 
> specify labels for nodes. For labels which may change over time, users will 
> have to start a cron job to update the labels. This has the following 
> limitations:
> - The cron job needs to be run in the YARN admin user.
> - This makes it a little complicate to maintain as users will have to make 
> sure this service/daemon is alive.
> Adding a Node Labels Provider in Resource Manager will provide user more 
> flexibility.

This message was sent by Atlassian JIRA

Reply via email to