Maybe throw the output of the startup of your job on a gist and share it and then we can maybe help some more.
The other thing to look at is how fast your mappers are completing. If they are completing in under let's say 10 seconds then the job tracker is "thrashing" and not able to allocate mappers fast enough. You could then adjust the input split siZe for your jibs a bit and slow the mappers down so the jobtracker can keep up James Sent from my mobile. Please excuse the typos. On 2011-05-02, at 12:08 AM, Guy Doulberg <[email protected]> wrote: > I know that... > Map allocations for a job is proportional to size of input data, > But in my case.... > I have a job that runs with 200 map slots.... > And a specific job that when it runs, the job tracker allocates to it 60 > mappers (although I think is should get more according to my calculations) > and 30 to the job that was already running. > What happened to the other 110 map slots? Why aren't they allocated? > > > Thanks, Guy > > > > -----Original Message----- > From: James Seigel [mailto:[email protected]] > Sent: Sunday, May 01, 2011 9:42 PM > To: [email protected] > Subject: Re: number of maps it lower than the cluster capacity > > Also an input split size thing in there as well. But definitely # of > mappers are proportional to input data size > > Sent from my mobile. Please excuse the typos. > > On 2011-05-01, at 11:26 AM, ShengChang Gu <[email protected]> wrote: > >> If I'm not mistaken,then the map slots = input data size / block size. >> >> 2011/5/2 Guy Doulberg <[email protected]> >> >>> Hey, >>> Maybe someone can give an idea where to look for the bug... >>> >>> I have a cluster with 270 slots for mappers, >>> And a fairSchedualer configured for it.... >>> >>> Sometimes this cluster allocates only 80 or 50 slots to the entire cluster. >>> Most of the time the most of slots are allocated . >>> >>> I noticed that there are special several jobs that cause the above >>> behavior, >>> >>> Thanks Guy, >>> >>> >>> >> >> >> -- >> 阿昌
