Sorry, the mail server formats my email.
Hi, all,I have configured "mapred.map.tasks" and "mapred.reduce.tasks" in
mapreduce.xml to "32"(see below), but Hadoop uses only 2 or 3 tasks when
running a job(see the attachment).How can I let Hadoop use more tasks so it can
run faster?
<property> <name>mapred.map.tasks</name> <value>32</value>
<description>define mapred.map tasks to be number of slave
hosts</description></property><property> <name>mapred.reduce.tasks</name>
<value>32</value> <description>define mapred.reduce tasks to be
number of slave hosts</description></property>
Dennis
--- On Tue, 10/5/10, Dennis <[email protected]> wrote:
From: Dennis <[email protected]>
Subject: map & reduce tasks numbers
To: [email protected]
Date: Tuesday, October 5, 2010, 6:31 PM
Hi, all,
I have configured "mapred.map.tasks" and "mapred.reduce.tasks" in mapreduce.xml
to "32"(see below), but Hadoop uses only 2 or 3 tasks when running a job(see
bellow).How can I let Hadoop use more tasks so it can run faster?
<property> <name>mapred.map.tasks</name> <value>32</value> <description>
define mapred.map tasks to be number of slave hosts </description></property>
<property> <name>mapred.reduce.tasks</name> <value>32</value> <description>
define mapred.reduce tasks to be number of slave
hosts </description></property>
JobidPriorityUserNameMap % CompleteMap TotalMaps CompletedReduce %
CompleteReduce TotalReduces CompletedJob Scheduling
Informationjob_201010051740_0001NORMALbillinject
dmoz100.00%22100.00%11NAjob_201010051740_0002NORMALbillcrawldb
crawl/crawldb100.00%33100.00%11NAjob_201010051740_0003NORMALbillgenerate:
select from
crawl/crawldb100.00%22100.00%11NAjob_201010051740_0004NORMALbillgenerate:
partition
crawl/segments/20101005180708100.00%22100.00%22NAjob_201010051740_0007NORMALbillinject
dmoz100.00%22100.00%11
Dennis
