Xiaodi Ke created YARN-3923:

             Summary: Introduce adaptive heartbeat between RM and AM
                 Key: YARN-3923
                 URL: https://issues.apache.org/jira/browse/YARN-3923
             Project: Hadoop YARN
          Issue Type: New Feature
          Components: api, applications, resourcemanager
            Reporter: Xiaodi Ke

Currently, the communication between RM and AM are based on pull-based 
heartbeat protocol. Along with AM heartbeat, AM sends the container requests to 
RM and retrieves the new allocated containers. Also, it will update RM's view 
of resource requests of applications. How frequently AM sends the heartbeat 
will impact the task throughput and latency of YARN scheduler.  Although the 
heartbeat interval can be configured in yarn-stie.xml (mapred-site.xml for MR 
AppMaster), if it is configured too short, it will increase the load of RM and 
bring unnecessary overhead. 

We propose the adaptive heartbeat between RM and AM to achieve a balance 
between updating AM's info promptly and minimizing the overhead of extra 
heartbeats. With adaptive heartbeat, AM still honors the current heartbeat 
interval and sends the heartbeat regularly. However, a heartbeat is triggered 
as soon as a container request is made. To prevent AM sending heartbeat too 
frequently, a minimum interval can be used to throttle the frequency of 

This message was sent by Atlassian JIRA

Reply via email to