[
https://issues.apache.org/jira/browse/YARN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14544244#comment-14544244
]
Karthik Kambatla commented on YARN-3630:
----------------------------------------
Number of events in the dispatcher is definitely a good indicator of how busy
the scheduler is. We could use it as one of the factors. Other factors:
# Some apps are more tolerant of longer intervals than others and we should
have a provision to specify an upper-limit on the interval.
# Weight/capacity of the queue based on whether it is FairScheduler/
CapacityScheduler
# How starved an application is. In FairScheduler, that translates to
min(fairshare - current-allocation, pending-resources).
Other things we need to consider:
# How do we plan to enforce the scheduler doesn't hear from applications sooner
than the specified interval? The AM can always choose to ignore, right?
# Are we considering automatically slowing down the NM heartbeats as well? With
continuous/asynchronous scheduling enabled, I suppose slowing down NM
heartbeats could be better than slowing down AM heartbeats. We should be
careful here though - we need to take into account how used a node is. If a
node is more allocated, slowing heartbeats could lead to delay in noticing
completed containers.
> YARN should suggest a heartbeat interval for applications
> ---------------------------------------------------------
>
> Key: YARN-3630
> URL: https://issues.apache.org/jira/browse/YARN-3630
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager, scheduler
> Affects Versions: 2.7.0
> Reporter: Zoltán Zvara
> Assignee: Xianyin Xin
> Priority: Minor
>
> It seems currently applications - for example Spark - are not adaptive to RM
> regarding heartbeat intervals. RM should be able to suggest a desired
> heartbeat interval to applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
