[ 
https://issues.apache.org/jira/browse/YARN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562211#comment-14562211
 ] 

Xianyin Xin commented on YARN-3630:
-----------------------------------

Thanks for your comments, [~vvasudev]!
{quote}
your patch doesn't check if the calculated interval is greater than the ping 
interval to determine liveliness for the AM and the NM. Is that by design?
{quote}
It's true we should do that. But in this patch I haven't add the mechanism of 
determining a up limit for the {{nextHeartbeatInterval}}. I think the limit 
should much less than the ping interval which is 10 minutes by default. Other 
hand, do you think a hard configurable limit is accepted? 
{quote}
With respect to adaptive heartbeats for the NMs - my concern is that the 
proposed solution will lead to behaviour where the NMs will be told to back off 
- the NMs will wait for sometime - the RM will receive a flood of NM updates - 
leading to the NMs being told to back off and so on and so forth. We'll end up 
in a situation where the pings will become clustered around particular time 
intervals, leading to container allocation and release delays. You might be 
better off picking a random interval between the default interval and the 
calculated interval to spread out the NM pings
{quote}
Thanks for reminding, it's a situation I didn't think much. I think your 
suggestion is a nice choice. 

> YARN should suggest a heartbeat interval for applications
> ---------------------------------------------------------
>
>                 Key: YARN-3630
>                 URL: https://issues.apache.org/jira/browse/YARN-3630
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager, scheduler
>    Affects Versions: 2.7.0
>            Reporter: Zoltán Zvara
>            Assignee: Xianyin Xin
>            Priority: Minor
>         Attachments: Notes_for_adaptive_heartbeat_policy.pdf, 
> YARN-3630.001.patch.patch, YARN-3630.002.patch
>
>
> It seems currently applications - for example Spark - are not adaptive to RM 
> regarding heartbeat intervals. RM should be able to suggest a desired 
> heartbeat interval to applications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to