Naganarasimha G R commented on YARN-2495:

        2.3. Everytime set the up-to-date labels to NodeHeartbeatRequest when 
do heartbeat.
#1 and #2 will all need add more fields in NodeHeartbeatRequest. I suggest to 
do in #3, it's more simple and we can improve it in further JIRA. 

I missed to discuss about this point in particular. If we do the 3rd approach, 
on every heartbeat in RM side we need to take either of the 2 approach 
        * Invoke getNodeLabelManager().replaceLabelsOnNode() which will 
validate the labels and then replace even though there is no change in the 
        * Resource tracker service validates the labels from the heartbeat with 
the NodeLabels manager and if modified Invoke 

Both of these approach are costlier because of following impacts :
        * if number of labels are more than the weight of heartbeat message 
will be more and if the cluster nodes are more than more network IO
        * checking of labels from Prev state to current state for all nodes is 
done in RM in earlier method each NM was taking care of it self.
        * Lot of read locks needs to be held @ NodesLabelsManager 

So based on these points would like to prefer option 1 as its minimal change 
and anyway we have already modified the request and response (proposed reject 
labels list)

 5. NodeStatusUpdaterImpl: 
* In RM ResourceTracker, if exception raise when replace labels on node, put 
the new labels to reject node labels to response.
* In NM NodeStatusUpdater, if reject node labels is not null, LOG.error 
rejected node labels, and print diagnostic message.
you meant "reject node labels is not empty" right based on comment2 null cannot 
be identified for repeated fields
Also as we either accept all labels or reject all labels can we have a flag 
whether RM accepted the labels or not ? and modify response proto when the 
NodeLabelsManager Interface changes ?

bq. 6) yarn_server_common_service_protos.proto
RegisterNodeManagerRequestProto and NodeHeartbeatRequestProto are modified in 
this file ... diff shows one line after the modification which looks like i 
have modified RegisterNodeManagerResponseProto

bq. 9 It no need to send shutdown message when any of the labels not accepted 
by RMNodeLabelsManager. Just add them to a reject node labels list, and add 
diagnostic message should be enough.
k this will take care but earlier i tried to send shutdown message because i 
wanted avoid the scenario where RM is confiugred for CentralNodeLabel and NM 
for distributed
But as per your earlier comment ??"PB cannot tell difference between null and 
empty for repeated fields."??. 
So do we need to do address this scenario by adding some boolean flag for 
DeCentralizedConfigEnable in RegisterNodeManagerRequestProto ?

> Allow admin specify labels in each NM (Distributed configuration)
> -----------------------------------------------------------------
>                 Key: YARN-2495
>                 URL: https://issues.apache.org/jira/browse/YARN-2495
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Naganarasimha G R
>         Attachments: YARN-2495.20141023-1.patch, YARN-2495.20141024-1.patch, 
> YARN-2495_20141022.1.patch
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml or using script 
> suggested by [~aw])
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels

This message was sent by Atlassian JIRA

Reply via email to