BTW, what I mentioned is fairsharepreemption not minimum share an alternative way to achieve that is to set minimum share of two queues to be equal(or other allocation scheme you like), and sum of them is equal to the capacity of the cluster, and enable minimumSharePreemption
Good Luck! Best, -- Nan Zhu School of Computer Science, McGill University On Wednesday, 16 January, 2013 at 11:43 AM, Nan Zhu wrote: > I think you should do that, so that when the allocation is inconsistent with > fair share, the tasks in the queue which occupies more beyond it's fair share > will be killed, and the available slots would be assigned to the other one > (assuming the weights of them are the same) > > Best, > > -- > Nan Zhu > School of Computer Science, > McGill University > > > > On Wednesday, 16 January, 2013 at 11:32 AM, Dhanasekaran Anbalagan wrote: > > > HI Nan, > > > > We have not enabled Fair Scheduler Preemption. > > > > -Dhanasekaran. > > > > Did I learn something today? If not, I wasted it. > > > > On Wed, Jan 16, 2013 at 11:21 AM, Nan Zhu <zhunans...@gmail.com > > (mailto:zhunans...@gmail.com)> wrote: > > > have you enabled task preemption? > > > > > > Best, > > > > > > -- > > > Nan Zhu > > > School of Computer Science, > > > McGill University > > > > > > > > > > > > On Wednesday, 16 January, 2013 at 10:45 AM, Justin Workman wrote: > > > > > > > Looks like weight for both pools is equal and all map slots are used. > > > > Therefore I don't believe anyone has priority for the next slots. Try > > > > setting research weight to 2. This should allow research to take slots > > > > as tech released them. > > > > > > > > Sent from my iPhone > > > > > > > > On Jan 16, 2013, at 8:26 AM, Dhanasekaran Anbalagan <bugcy...@gmail.com > > > > (mailto:bugcy...@gmail.com)> wrote: > > > > > > > > > HI Guys > > > > > > > > > > We configured fair scheduler with cdh4, Fair scheduler not work > > > > > properly. > > > > > Map Task Capacity = 1380 > > > > > Reduce Task Capacity = 720 > > > > > > > > > > We create two users tech and research, we configured equal weight 1 > > > > > But, I stared job in research user mapper will not allocated why? > > > > > please guide me guys. > > > > > > > > > > <?xml version="1.0"?> > > > > > <allocations> > > > > > <pool name="tech"> > > > > > <minMaps>5</minMaps> > > > > > <minReduces>5</minReduces> > > > > > <maxRunningJobs>30</maxRunningJobs> > > > > > <weight>1.0</weight> > > > > > </pool> > > > > > <pool name="research"> > > > > > <minMaps>5</minMaps> > > > > > <minReduces>5</minReduces> > > > > > <maxRunningJobs>30</maxRunningJobs> > > > > > <weight>1.0</weight> > > > > > </pool> > > > > > </allocations> > > > > > > > > > > Note: we have tested with Hadoop Stream job. > > > > > > > > > > Fair Scheduler Administration > > > > > Pools > > > > > Pool > > > > > Running Jobs > > > > > Map Tasks > > > > > Reduce Tasks > > > > > Scheduling Mode > > > > > > > > > > Min Share > > > > > Max Share > > > > > Running > > > > > Fair Share > > > > > Min Share > > > > > Max Share > > > > > Running > > > > > Fair Share > > > > > > > > > > research > > > > > 1 > > > > > 5 > > > > > - > > > > > 90 > > > > > 690.0 > > > > > 5 > > > > > - > > > > > 0 > > > > > 0.0 > > > > > FAIR > > > > > > > > > > tech > > > > > 3 > > > > > 5 > > > > > - > > > > > 1266 > > > > > 690.0 > > > > > 5 > > > > > - > > > > > 24 > > > > > 24.0 > > > > > FAIR > > > > > > > > > > default > > > > > 0 > > > > > 0 > > > > > - > > > > > 0 > > > > > 0.0 > > > > > 0 > > > > > - > > > > > 0 > > > > > 0.0 > > > > > FAIR > > > > > > > > > > > > > > > > > > > > Running Jobs > > > > > Submitted > > > > > JobID > > > > > User > > > > > Name > > > > > Pool > > > > > Priority > > > > > Map Tasks > > > > > Reduce Tasks > > > > > > > > > > Finished > > > > > Running > > > > > Fair Share > > > > > Weight > > > > > Finished > > > > > Running > > > > > Fair Share > > > > > Weight > > > > > > > > > > Jan 16, 08:51 > > > > > job_201301071639_2118 > > > > > (http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2118) > > > > > tech > > > > > streamjob5335328828469969152.jar > > > > > > > > > > > > > > > 30466 / 53724 (tel:30466%20%2F%2053724) > > > > > 583 > > > > > 313.5 > > > > > 1.0 > > > > > 0 / 24 > > > > > 0 > > > > > 0.0 > > > > > 1.0 > > > > > > > > > > Jan 16, 09:56 > > > > > job_201301071639_2147 > > > > > (http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2147) > > > > > research > > > > > streamjob8832181817213433660.jar > > > > > > > > > > > > > > > 4175 / 9581 > > > > > 90 > > > > > 690.0 > > > > > 1.0 > > > > > 0 / 24 > > > > > 0 > > > > > 0.0 > > > > > 1.0 > > > > > > > > > > Jan 16, 10:01 > > > > > job_201301071639_2148 > > > > > (http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2148) > > > > > tech > > > > > streamjob8773848575543653055.jar > > > > > > > > > > > > > > > 1842 / 15484 > > > > > 620 > > > > > 313.5 > > > > > 1.0 > > > > > 0 / 24 > > > > > 0 > > > > > 0.0 > > > > > 1.0 > > > > > > > > > > Jan 16, 10:08 > > > > > job_201301071639_2155 > > > > > (http://172.16.30.122:50030/jobdetails.jsp?jobid=job_201301071639_2155) > > > > > tech > > > > > counterfactualsim-prod.eagle-EagleDepthSignalDisabled-prod.eagle > > > > > > > > > > > > > > > 387 / 450 > > > > > 63 > > > > > 63.0 > > > > > 1.0 > > > > > 0 / 24 > > > > > 24 > > > > > 24.0 > > > > > 1.0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > >
