Varun Vasudev commented on YARN-3922:

[~xiaodike] - can you take a look at YARN-3630? It also discusses similar 

>  Introduce adaptive heartbeat between RM and NM
> -----------------------------------------------
>                 Key: YARN-3922
>                 URL: https://issues.apache.org/jira/browse/YARN-3922
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, resourcemanager
>            Reporter: Xiaodi Ke
> Currently, the communication between RM and NM are based on pull-based 
> heartbeat protocol. Along with the NM heartbeat, it updates the status of 
> containers (i.e. FINISHED container). This also updates the RM’s view of 
> available resource and triggers scheduling. How frequently the NM sends the 
> heartbeat will impact the task throughput and latency of YARN scheduler.  
> Although the heartbeat interval can be configured in yarn-stie.xml, it will 
> increase the load of RM and bring unnecessary overhead if the interval is 
> configured too short. 
> We propose the adaptive heartbeat between RM and NM to achieve a balance 
> between updating NM’s info promptly and minimizing the overhead of extra 
> heartbeats. With adaptive heartbeat, NM still honors the current heartbeat 
> interval and sends the heartbeat regularly. However, a heartbeat will be 
> triggered as soon as any container status is changed.  Also a minimum 
> interval can be configured to prevent NM from sending heartbeat too 
> frequently.

This message was sent by Atlassian JIRA

Reply via email to