[
https://issues.apache.org/jira/browse/YARN-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14184133#comment-14184133
]
Naganarasimha G R commented on YARN-2495:
-----------------------------------------
{quote}
2.NodeHeartbeatRequestPBImpl:
2.3. Everytime set the up-to-date labels to NodeHeartbeatRequest when
do heartbeat.
#1 and #2 will all need add more fields in NodeHeartbeatRequest. I suggest to
do in #3, it's more simple and we can improve it in further JIRA.
{quote}
I missed to discuss about this point in particular. If we do the 3rd approach,
on every heartbeat in RM side we need to take either of the 2 approach
* Invoke getNodeLabelManager().replaceLabelsOnNode() which will
validate the labels and then replace even though there is no change in the
labels
* Resource tracker service validates the labels from the heartbeat with
the NodeLabels manager and if modified Invoke
getNodeLabelManager().replaceLabelsOnNode()
Both of these approach are costlier because of following impacts :
* if number of labels are more than the weight of heartbeat message
will be more and if the cluster nodes are more than more network IO
* checking of labels from Prev state to current state for all nodes is
done in RM in earlier method each NM was taking care of it self.
* Lot of read locks needs to be held @ NodesLabelsManager
So based on these points would like to prefer option 1 as its minimal change
and anyway we have already modified the request and response (proposed reject
labels list)
{quote}
5. NodeStatusUpdaterImpl:
* In RM ResourceTracker, if exception raise when replace labels on node, put
the new labels to reject node labels to response.
* In NM NodeStatusUpdater, if reject node labels is not null, LOG.error
rejected node labels, and print diagnostic message.
{quote}
you meant "reject node labels is not empty" right based on comment2 null cannot
be identified for repeated fields
Also as we either accept all labels or reject all labels can we have a flag
whether RM accepted the labels or not ? and modify response proto when the
NodeLabelsManager Interface changes ?
bq. 6) yarn_server_common_service_protos.proto
RegisterNodeManagerRequestProto and NodeHeartbeatRequestProto are modified in
this file ... diff shows one line after the modification which looks like i
have modified RegisterNodeManagerResponseProto
bq. 9 It no need to send shutdown message when any of the labels not accepted
by RMNodeLabelsManager. Just add them to a reject node labels list, and add
diagnostic message should be enough.
k this will take care but earlier i tried to send shutdown message because i
wanted avoid the scenario where RM is confiugred for CentralNodeLabel and NM
for distributed
But as per your earlier comment ??"PB cannot tell difference between null and
empty for repeated fields."??.
So do we need to do address this scenario by adding some boolean flag for
DeCentralizedConfigEnable in RegisterNodeManagerRequestProto ?
> Allow admin specify labels in each NM (Distributed configuration)
> -----------------------------------------------------------------
>
> Key: YARN-2495
> URL: https://issues.apache.org/jira/browse/YARN-2495
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Reporter: Wangda Tan
> Assignee: Naganarasimha G R
> Attachments: YARN-2495.20141023-1.patch, YARN-2495.20141024-1.patch,
> YARN-2495_20141022.1.patch
>
>
> Target of this JIRA is to allow admin specify labels in each NM, this covers
> - User can set labels in each NM (by setting yarn-site.xml or using script
> suggested by [~aw])
> - NM will send labels to RM via ResourceTracker API
> - RM will set labels in NodeLabelManager when NM register/update labels
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
