Ah my bad that I missed it when reading your doc. Yes that property being true would make it only LOG about preemption scenarios, not do preemption.
On Wed, Mar 7, 2012 at 12:05 PM, hao.wang <hao.w...@ipinyou.com> wrote: > Hi, Thanks for your reply! > I have solved this problem by setting > "mapred.fairscheduler.preemption.only.log " to "false". The preemption > works! > But I don't know why can not set "mapred.fairscheduler.preemption.only.log " > to "true". Is it a bug? > > regards, > > > > 2012-03-07 > ________________________________ > hao.wang > ________________________________ > 发件人： Harsh J > 发送时间： 2012-03-07 14:14:05 > 收件人： common-user > 抄送： > 主题： Re: Fair Scheduler Problem > Hello Hao, > Its best to submit CDH user queries to > https://groups.google.com/a/cloudera.org/group/cdh-user/topics > (cdh-u...@cloudera.org) where the majority of CDH users community > resides. > How do you determine that preemption did not/does not work? Preemption > between pools occurs if a pool's minShare isn't satisfied within > preemption-timeout seconds. In this case, it will preempt tasks from > other pools. > Your settings look alright on a high level. Does your log not carry > any preemption logs? What was your pool's share scenario when you > tried to observe if it works or not? > On Wed, Mar 7, 2012 at 8:35 AM, hao.wang <hao.w...@ipinyou.com> wrote: >> Hi ,All, >> I encountered a problem in using Cloudera Hadoop 0.20.2-cdh3u1. When I >> use the fair Scheduler I find the scheduler seems not support preemption. >> Can anybody tell me whether preemption is supported in this version? >> This is my configration: >> mapred-site.xml >> <property> >> <name>mapred.jobtracker.taskScheduler</name> >> <value>org.apache.hadoop.mapred.FairScheduler</value> >> </property> >> <property> >> <name>mapred.fairscheduler.allocation.file</name> >> <value>/usr/lib/hadoop-0.20/conf/fair-scheduler.xml</value> >> </property> >> <property> >> <name>mapred.fairscheduler.preemption</name> >> <value>true</value> >> </property> >> <property> >> <name>mapred.fairscheduler.preemption.only.log</name> >> <value>true</value> >> </property> >> <property> >> <name>mapred.fairscheduler.preemption.interval</name> >> <value>15000</value> >> </property> >> <property> >> <name>mapred.fairscheduler.weightadjuster</name> >> <value>org.apache.hadoop.mapred.NewJobWeightBooster</value> >> </property> >> <property> >> <name>mapred.fairscheduler.sizebasedweight</name> >> <value>true</value> >> </property> >> fair-scheduler.xml >> <allocations> >> <pool name="root"> >> <minMaps>10</minMaps> >> <minReduces>5</minReduces> >> <maxMaps>200</maxMaps> >> <maxReduces>80</maxReduces> >> <maxRunningJobs>100</maxRunningJobs> >> <minSharePreemptionTimeout>30</minSharePreemptionTimeout> >> <weight>1.0</weight> >> </pool> >> <pool name="hadoop"> >> <minMaps>10</minMaps> >> <minReduces>5</minReduces> >> <maxMaps>80</maxMaps> >> <maxReduces>80</maxReduces> >> <maxRunningJobs>5</maxRunningJobs> >> <minSharePreemptionTimeout>30</minSharePreemptionTimeout> >> <weight>1.0</weight> >> </pool> >> <user name="user1"> >> <maxRunningJobs>10</maxRunningJobs> >> </user> >> <poolMaxJobsDefault>20</poolMaxJobsDefault> >> <userMaxJobsDefault>10</userMaxJobsDefault> >> <defaultMinSharePreemptionTimeout>30</defaultMinSharePreemptionTimeout> >> <fairSharePreemptionTimeout>30</fairSharePreemptionTimeout> >> </allocations> >> >> regards, >> >> 2012-03-07 >> >> >> >> hao.wang > -- > Harsh J -- Harsh J
