Kannan Rajah resolved YARN-2989.
    Resolution: Not a Problem

> Better Load Balancing in Fair Scheduler
> ---------------------------------------
>                 Key: YARN-2989
>                 URL: https://issues.apache.org/jira/browse/YARN-2989
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>    Affects Versions: 2.5.0
>            Reporter: Kannan Rajah
> While porting Fair Scheduler from MR1, we seem to have changed the logic 
> behind task distribution across nodes (MAPREDUCE-3451).
> In MR1, a load factor was computed using runnableMaps/totalMapSlots and this 
> was used to determine how many tasks need to be given to a node such that the 
> overall cluster load is evenly distributed. In one heartbeat, we could assign 
> multiple tasks. In YARN, we have the option to assign multiple tasks to a 
> node, but this is disabled by default (YARN-302). Even when it is enabled, 
> the number of tasks to assign is statically configured. So it won't ensure 
> that load is evenly distributed. Why not bring back the load factor based 
> check? Any reason why it was not done? This is actually more relevant with 
> label based scheduling.
> If there are no objections, I would like to implement it for both normal and 
> label based scheduling scenarios.

This message was sent by Atlassian JIRA

Reply via email to