[
https://issues.apache.org/jira/browse/YARN-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kannan Rajah resolved YARN-2989.
--------------------------------
Resolution: Not a Problem
> Better Load Balancing in Fair Scheduler
> ---------------------------------------
>
> Key: YARN-2989
> URL: https://issues.apache.org/jira/browse/YARN-2989
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Affects Versions: 2.5.0
> Reporter: Kannan Rajah
>
> While porting Fair Scheduler from MR1, we seem to have changed the logic
> behind task distribution across nodes (MAPREDUCE-3451).
> In MR1, a load factor was computed using runnableMaps/totalMapSlots and this
> was used to determine how many tasks need to be given to a node such that the
> overall cluster load is evenly distributed. In one heartbeat, we could assign
> multiple tasks. In YARN, we have the option to assign multiple tasks to a
> node, but this is disabled by default (YARN-302). Even when it is enabled,
> the number of tasks to assign is statically configured. So it won't ensure
> that load is evenly distributed. Why not bring back the load factor based
> check? Any reason why it was not done? This is actually more relevant with
> label based scheduling.
> If there are no objections, I would like to implement it for both normal and
> label based scheduling scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
