Hi, In order to have different number of map tasks for each of the jobs, in the run method of the code , I had the following syntax:
conf.setNumMapTasks(num); // for number of map tasks conf.setNumReduceTasks(num); // for number of reduce tasks conf is the JobConf object and num is the number of map tasks/reduce tasks that are to be had for the job. It worked for me, hope it works for you. :) btw, i'm using hadoop-0.17.0. I'm not sure if there is any deprecation of these methods in later versions. V.V.Chaitanya Krishna IIIT,Hyderabad India On Fri, Oct 17, 2008 at 1:25 AM, Steve Gao <[EMAIL PROTECTED]> wrote: > > Would anybody help me? > Can I use > -jobconf mapred.map.task=50 in streaming command to change the job's number > of mappers? > > I don't have a hadoop at hand and can not verify it. Thanks for your help. > > --- On Wed, 10/15/08, Steve Gao <[EMAIL PROTECTED]> wrote: > From: Steve Gao <[EMAIL PROTECTED]> > Subject: How to change number of mappers in Hadoop streaming? > To: [EMAIL PROTECTED] > Cc: [email protected] > Date: Wednesday, October 15, 2008, 7:25 PM > > Is there a way to change number of mappers in Hadoop streaming command > line? > I know I can change hadoop-default.xml: > > <property> > <name>mapred.map.tasks</name> > <value>10</value> > <description>The default number of map tasks per job. Typically set > to a prime several times greater than number of available hosts. > Ignored when mapred.job.tracker is "local". > </description> > </property> > > But that's for all jobs. What if I just want each job has different > NUM_OF_Mappers themselves? Thanks > > > > > > > >
