I forgot to mention that although the number of map tasks are set in the code as I mentioned before, the actual number of map tasks are not essentially the same number but is very close to this number.
V.V.Chaitanya Krishna IIIT,Hyderabad India On Sun, Oct 26, 2008 at 4:29 PM, chaitanya krishna < [EMAIL PROTECTED]> wrote: > Hi, > > In order to have different number of map tasks for each of the jobs, in > the run method of the code , I had the following syntax: > > conf.setNumMapTasks(num); // for number of map tasks > > conf.setNumReduceTasks(num); // for number of reduce tasks > > conf is the JobConf object and num is the number of map tasks/reduce tasks > that are to be had for the job. It worked for me, hope it works for you. :) > > btw, i'm using hadoop-0.17.0. I'm not sure if there is any deprecation of > these methods in later versions. > > > V.V.Chaitanya Krishna > IIIT,Hyderabad > India > > > On Fri, Oct 17, 2008 at 1:25 AM, Steve Gao <[EMAIL PROTECTED]> wrote: > >> >> Would anybody help me? >> Can I use >> -jobconf mapred.map.task=50 in streaming command to change the job's >> number of mappers? >> >> I don't have a hadoop at hand and can not verify it. Thanks for your help. >> >> --- On Wed, 10/15/08, Steve Gao <[EMAIL PROTECTED]> wrote: >> From: Steve Gao <[EMAIL PROTECTED]> >> Subject: How to change number of mappers in Hadoop streaming? >> To: [email protected] >> Cc: [EMAIL PROTECTED] >> Date: Wednesday, October 15, 2008, 7:25 PM >> >> Is there a way to change number of mappers in Hadoop streaming command >> line? >> I know I can change hadoop-default.xml: >> >> <property> >> <name>mapred.map.tasks</name> >> <value>10</value> >> <description>The default number of map tasks per job. Typically set >> to a prime several times greater than number of available hosts. >> Ignored when mapred.job.tracker is "local". >> </description> >> </property> >> >> But that's for all jobs. What if I just want each job has different >> NUM_OF_Mappers themselves? Thanks >> >> >> >> >> >> >> >> > > >
