Naganarasimha G R commented on YARN-2923:

Thanks [~leftnoteasy], for the feedback, but had few things to discuss.
bq. 1) All script provider related configurations/logic should be removed. 
Will take care of it, earlier thought at-least white-list entities let it be 
there but not a problem will correct it.

PreviousNodeLabels will be reset every time if we do a fetch. (To avoid handle 
same node labels as much as possible)
Don't do check if new fetched node label is as same as previousNodeLabels. 
(Also, avoid handle same node label)
May be I dint get your approach completely here ? but based on earlier 
discussions in 2495 we had finalized that we will send the node labels only 
when there is a change because 
# unnecessary Network traffic which is amplified in a large cluster which has 
lot of NM's
# Either RM needs to do the validation of whether the labels are modified or 
blindly update the NodeLabelsStore. In former case is not better because RM 
will be overloaded to do this operation and in later case unnecessary Store 
file updates, i feel the magnitude of updates which will be happening in the 
large cluster with heartbeat coming in every second is too high and all are 
unwanted updates.
Based on the above points i feel its better to check and update RM only when 
there is a change in the NM's node labels

bq. Don't reset node label if fetched node label is incorrect. (This should be 
a part of error handling, we should treat it's a error to be avoided instead of 
force reset it)
IIUC this was the conclusion we had in YARN-2495 in the following 
 which i feel is correct because assume a case where in label is based on java 
version(/lib version) and java version is upgraded in a given NM but the 
centralized labels new java version is not yet added then NM will fail to 
update the node labels but if we do not reset it, RM will maintain the NM's 
NodeLabel as of Prev java version and the client app if run on this node it 
might fail due to this.

bq. A little cosmetic suggestion.
Hmm agree on this but approach i felt we can have a inner class like 
{{NMDistributedNodeLabelsHandler}} and that can maintain the state of previous 
labels and takes care of initializing and taking care of all methods and 
logging related to NodeLabels . I have already added some methods but we 
require to maintain some state of labels in the heartbeat flow so introducing a 
class like this and pushing all methods related to labels there would be much 
better and more readable, Thoughts ?

bq. I suggest to keep provider within nodemanager 

> Support configuration based NodeLabelsProvider Service in Distributed Node 
> Label Configuration Setup 
> -----------------------------------------------------------------------------------------------------
>                 Key: YARN-2923
>                 URL: https://issues.apache.org/jira/browse/YARN-2923
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>             Fix For: 2.8.0
>         Attachments: YARN-2923.20141204-1.patch, YARN-2923.20141210-1.patch, 
> YARN-2923.20150328-1.patch, YARN-2923.20150404-1.patch, 
> YARN-2923.20150517-1.patch
> As part of Distributed Node Labels configuration we need to support Node 
> labels to be configured in Yarn-site.xml. And on modification of Node Labels 
> configuration in yarn-site.xml, NM should be able to get modified Node labels 
> from this NodeLabelsprovider service without NM restart

This message was sent by Atlassian JIRA

Reply via email to