Xiaodi Ke created YARN-3923:
-------------------------------
Summary: Introduce adaptive heartbeat between RM and AM
Key: YARN-3923
URL: https://issues.apache.org/jira/browse/YARN-3923
Project: Hadoop YARN
Issue Type: New Feature
Components: api, applications, resourcemanager
Reporter: Xiaodi Ke
Currently, the communication between RM and AM are based on pull-based
heartbeat protocol. Along with AM heartbeat, AM sends the container requests to
RM and retrieves the new allocated containers. Also, it will update RM's view
of resource requests of applications. How frequently AM sends the heartbeat
will impact the task throughput and latency of YARN scheduler. Although the
heartbeat interval can be configured in yarn-stie.xml (mapred-site.xml for MR
AppMaster), if it is configured too short, it will increase the load of RM and
bring unnecessary overhead.
We propose the adaptive heartbeat between RM and AM to achieve a balance
between updating AM's info promptly and minimizing the overhead of extra
heartbeats. With adaptive heartbeat, AM still honors the current heartbeat
interval and sends the heartbeat regularly. However, a heartbeat is triggered
as soon as a container request is made. To prevent AM sending heartbeat too
frequently, a minimum interval can be used to throttle the frequency of
heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
