Xiaodi Ke commented on YARN-3923:

Hi [~vvasudev], please see my reply in YARN-3922. 

> Introduce adaptive heartbeat between RM and AM
> ----------------------------------------------
>                 Key: YARN-3923
>                 URL: https://issues.apache.org/jira/browse/YARN-3923
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: api, applications, resourcemanager
>            Reporter: Xiaodi Ke
>              Labels: newbie
> Currently, the communication between RM and AM are based on pull-based 
> heartbeat protocol. Along with AM heartbeat, AM sends the container requests 
> to RM and retrieves the new allocated containers. Also, it will update RM's 
> view of resource requests of applications. How frequently AM sends the 
> heartbeat will impact the task throughput and latency of YARN scheduler.  
> Although the heartbeat interval can be configured in yarn-stie.xml 
> (mapred-site.xml for MR AppMaster), if it is configured too short, it will 
> increase the load of RM and bring unnecessary overhead. 
> We propose the adaptive heartbeat between RM and AM to achieve a balance 
> between updating AM's info promptly and minimizing the overhead of extra 
> heartbeats. With adaptive heartbeat, AM still honors the current heartbeat 
> interval and sends the heartbeat regularly. However, a heartbeat is triggered 
> as soon as a container request is made. To prevent AM sending heartbeat too 
> frequently, a minimum interval can be used to throttle the frequency of 
> heartbeat.

This message was sent by Atlassian JIRA

Reply via email to