[
https://issues.apache.org/jira/browse/YARN-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257622#comment-14257622
]
Karthik Kambatla commented on YARN-2989:
----------------------------------------
[~rkannan82] - as you might have noticed, we want to move towards continuous
scheduling being enabled by default. One shouldn't need assignMultiple with
continuous scheduling.
> Better Load Balancing in Fair Scheduler
> ---------------------------------------
>
> Key: YARN-2989
> URL: https://issues.apache.org/jira/browse/YARN-2989
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Affects Versions: 2.5.0
> Reporter: Kannan Rajah
>
> While porting Fair Scheduler from MR1, we seem to have changed the logic
> behind task distribution across nodes (MAPREDUCE-3451).
> In MR1, a load factor was computed using runnableMaps/totalMapSlots and this
> was used to determine how many tasks need to be given to a node such that the
> overall cluster load is evenly distributed. In one heartbeat, we could assign
> multiple tasks. In YARN, we have the option to assign multiple tasks to a
> node, but this is disabled by default (YARN-302). Even when it is enabled,
> the number of tasks to assign is statically configured. So it won't ensure
> that load is evenly distributed. Why not bring back the load factor based
> check? Any reason why it was not done? This is actually more relevant with
> label based scheduling.
> If there are no objections, I would like to implement it for both normal and
> label based scheduling scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
