[ https://issues.apache.org/jira/browse/YARN-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626527#comment-14626527 ]
Xiaodi Ke commented on YARN-3922: --------------------------------- We already have some ideas and are working on the prototype and design doc. We will post the design doc and patch once it is ready. > Introduce adaptive heartbeat between RM and NM > ----------------------------------------------- > > Key: YARN-3922 > URL: https://issues.apache.org/jira/browse/YARN-3922 > Project: Hadoop YARN > Issue Type: New Feature > Components: nodemanager, resourcemanager > Reporter: Xiaodi Ke > > Currently, the communication between RM and NM are based on pull-based > heartbeat protocol. Along with the NM heartbeat, it updates the status of > containers (i.e. FINISHED container). This also updates the RM’s view of > available resource and triggers scheduling. How frequently the NM sends the > heartbeat will impact the task throughput and latency of YARN scheduler. > Although the heartbeat interval can be configured in yarn-stie.xml, it will > increase the load of RM and bring unnecessary overhead if the interval is > configured too short. > We propose the adaptive heartbeat between RM and NM to achieve a balance > between updating NM’s info promptly and minimizing the overhead of extra > heartbeats. With adaptive heartbeat, NM still honors the current heartbeat > interval and sends the heartbeat regularly. However, a heartbeat will be > triggered as soon as any container status is changed. Also a minimum > interval can be configured to prevent NM from sending heartbeat too > frequently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)