Varun Saxena commented on YARN-3644:

Thanks [~raju.bairishetti] for working on this.

FTew comments :
# No need to override {{MyNodeManager3#getNodeStatusUpdater}} and 
{{MyNodeManager3#serviceStop}}. serviceStop for instance only calls 
# {{MyNodeStatusUpdater6#context}} is not required.
# The test doesnt really check for whether ConnectionException was thrown or NM 
Shutdown event was called or not. I think you check whether the flow has been 
hit or not using Mockito mock or spy. For instance, if you call 
{{NodeManager#start}}, service state will be STARTED irrespective of whether 
code written by you has been hit or not.
# I think log added can be logged at WARN level instead of ERROR.
# Also the log says "Not shutting down NodeManager. Retry after default 
heartbeat interval time". We can instead say something like "Unable to connect 
to RM...Retry after default heartbeat time".
# The config name is {{yarn.nodemanager.shutdown.on.RM.connection.failures}}. 
All our config names are in lowercase, just for the sake of consistency, maybe 
RM can be in lowercase too. Thoughts ? 

> Node manager shuts down if unable to connect with RM
> ----------------------------------------------------
>                 Key: YARN-3644
>                 URL: https://issues.apache.org/jira/browse/YARN-3644
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Srikanth Sundarrajan
>            Assignee: Raju Bairishetti
>         Attachments: YARN-3644.001.patch, YARN-3644.001.patch, 
> YARN-3644.002.patch, YARN-3644.003.patch, YARN-3644.patch
> When NM is unable to connect to RM, NM shuts itself down.
> {code}
>           } catch (ConnectException e) {
>             //catch and throw the exception if tried MAX wait time to connect 
> RM
>             dispatcher.getEventHandler().handle(
>                 new NodeManagerEvent(NodeManagerEventType.SHUTDOWN));
>             throw new YarnRuntimeException(e);
> {code}
> In large clusters, if RM is down for maintenance for longer period, all the 
> NMs shuts themselves down, requiring additional work to bring up the NMs.
> Setting the yarn.resourcemanager.connect.wait-ms to -1 has other side 
> effects, where non connection failures are being retried infinitely by all 
> YarnClients (via RMProxy).

This message was sent by Atlassian JIRA

Reply via email to