[ https://issues.apache.org/jira/browse/YARN-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kannan Rajah resolved YARN-2989. -------------------------------- Resolution: Not a Problem > Better Load Balancing in Fair Scheduler > --------------------------------------- > > Key: YARN-2989 > URL: https://issues.apache.org/jira/browse/YARN-2989 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler > Affects Versions: 2.5.0 > Reporter: Kannan Rajah > > While porting Fair Scheduler from MR1, we seem to have changed the logic > behind task distribution across nodes (MAPREDUCE-3451). > In MR1, a load factor was computed using runnableMaps/totalMapSlots and this > was used to determine how many tasks need to be given to a node such that the > overall cluster load is evenly distributed. In one heartbeat, we could assign > multiple tasks. In YARN, we have the option to assign multiple tasks to a > node, but this is disabled by default (YARN-302). Even when it is enabled, > the number of tasks to assign is statically configured. So it won't ensure > that load is evenly distributed. Why not bring back the load factor based > check? Any reason why it was not done? This is actually more relevant with > label based scheduling. > If there are no objections, I would like to implement it for both normal and > label based scheduling scenarios. -- This message was sent by Atlassian JIRA (v6.3.4#6332)