Thanks, Ahmad, But I'am using Nutch 1.1, and I installed it following this: http://wiki.apache.org/nutch/NutchHadoopTutorial. Besides, the "crawldb crawl/crawldb" job uses more mapreduce tasks, usually 4, while other jobs uses only 2. Any idea?
Dennis --- On Tue, 10/5/10, Ahmad Al-Amri <[email protected]> wrote: From: Ahmad Al-Amri <[email protected]> Subject: Re: need a larger map task number To: [email protected] Date: Tuesday, October 5, 2010, 10:01 PM Hello, "check wether your "Working directory"(Run -> Run Configurations -> Tab Arguments -> Working Directory) points to the Nutch base directory (where your conf/nucht-site.xml is located)." http://search-lucene.com/m/xwpG02rotOD1/v=threaded you can see hadoop.log to get what is the error exactly Regards; Ahmad ________________________________ From: Steve Cohen <[email protected]> To: [email protected] Sent: Tue, October 5, 2010 4:40:22 PM Subject: Re: need a larger map task number For nutch, I found that updating the values in hadoop-site.xml was enough, though I also set values for mapred.tasktracker.map.tasks.maximum and mapred.tasktracker.reduce.tasks.maximum. On Tue, Oct 5, 2010 at 9:24 AM, Dennis <[email protected]> wrote: > Hi, all > My "fetch" job uses only 2 map tasks and 2 reduce tasks although I > configured "mapred.map.tasks" and "mapred.reduce.tasks" in "mapreduce.xml" > to "32", while I need it run faster.How can I make nutch to use more map and > reduce tasks when it's fetching? > Dennis > > >
