[
https://issues.apache.org/jira/browse/YARN-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782897#comment-16782897
]
Wilfred Spiegelenburg commented on YARN-6487:
---------------------------------------------
It works both ways:
Before we can schedule we need to update the current usage and shares etc. This
runs in an update thread. Continuous scheduling triggers that update. The
heartbeats when they are processed do the same. This updating requires a lock
of the scheduler as does the scheduling process itself. The extra update demand
is the trigger/
So you get into a state that the heartbeat, the updates and the scheduling
itself are all waiting for the lock. The larger the number of nodes, the larger
the number of applications is (in most cases) and the larger the number of
queues (again in most cases). All this combined causes processing to start
lagging and the continuous scheduling really loses its function.
Node numbers influence continuous scheduling and the other way around.
> FairScheduler: remove continuous scheduling (YARN-1010)
> -------------------------------------------------------
>
> Key: YARN-6487
> URL: https://issues.apache.org/jira/browse/YARN-6487
> Project: Hadoop YARN
> Issue Type: Task
> Components: fairscheduler
> Affects Versions: 2.7.0
> Reporter: Wilfred Spiegelenburg
> Assignee: Wilfred Spiegelenburg
> Priority: Major
>
> Remove deprecated FairScheduler continuous scheduler code
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
