Xiaodi Ke created YARN-3922:
-------------------------------
Summary: Introduce adaptive heartbeat between RM and NM
Key: YARN-3922
URL: https://issues.apache.org/jira/browse/YARN-3922
Project: Hadoop YARN
Issue Type: New Feature
Components: nodemanager, resourcemanager
Reporter: Xiaodi Ke
Currently, the communication between RM and NM are based on pull-based
heartbeat protocol. Along with the NM heartbeat, it updates the status of
containers (i.e. FINISHED container). This also updates the RM’s view of
available resource and triggers scheduling. How frequently the NM sends the
heartbeat will impact the task throughput and latency of YARN scheduler.
Although the heartbeat interval can be configured in yarn-stie.xml, it will
increase the load of RM and bring unnecessary overhead if the interval is
configured too short.
We propose the adaptive heartbeat between RM and NM to achieve a balance
between updating NM’s info promptly and minimizing the overhead of extra
heartbeats. With adaptive heartbeat, NM still honors the current heartbeat
interval and sends the heartbeat regularly. However, a heartbeat will be
triggered as soon as any container status is changed. Also a minimum interval
can be configured to prevent NM from sending heartbeat too frequently.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
