Perhaps you can try YARN-80's yarn.scheduler.capacity.node-locality-delay set to 1 for your CapacityScheduler (are you using this or the Fifo)?
On Fri, Dec 14, 2012 at 1:37 AM, lohit <[email protected]> wrote: > I am running hadoop-2.0.3 alpha branch. > Right now I am trying TestDFSIO, where I would like to have even > distribution of tasks. > As soon as I submit job RM tends to allocate as many containers as possible > from subset of nodes. > > 2012/12/13 Thomas Graves <[email protected]> > >> What kind of job are you running? What version of YARN? >> >> Tom >> >> On 12/13/12 12:24 AM, "lohit" <[email protected]> wrote: >> >> >Hi Devs, >> > >> >I am running few experiments on YARN. On a 50 node cluster I run only one >> >job. >> >If I kick off job with 150 tasks, I would expect on an average 3 tasks in >> >containers per node. >> >But I see many nodes end up having 29-30 containers spawned and many >> >without any tasks at all. >> >Do I have to use some kind of parameter (delayed scheduling) to achieve >> >even distribution? >> > >> >-- >> >Have a Nice Day! >> >Lohit >> >> > > > -- > Have a Nice Day! > Lohit -- Harsh J
