Hello Dennis; This have to be a response to another post.
Sorry for this :) Regards; ________________________________ From: Dennis <[email protected]> To: [email protected] Cc: [email protected] Sent: Wed, October 6, 2010 3:49:30 AM Subject: Re: need a larger map task number Thanks, Ahmad, But I'am using Nutch 1.1, and I installed it following this: http://wiki.apache.org/nutch/NutchHadoopTutorial. Besides, the "crawldb crawl/crawldb" job uses more mapreduce tasks, usually 4, while other jobs uses only 2. Any idea? Dennis --- On Tue, 10/5/10, Ahmad Al-Amri <[email protected]> wrote: >From: Ahmad Al-Amri <[email protected]> >Subject: Re: need a larger map task number >To: [email protected] >Date: Tuesday, October 5, 2010, 10:01 PM > > >Hello, > > >"check wether your "Working directory"(Run -> Run Configurations -> Tab >Arguments -> Working Directory) points to the Nutch base directory (where >your conf/nucht-site.xml is located)." >http://search-lucene.com/m/xwpG02rotOD1/v=threaded > >you can see hadoop.log to get what is the error exactly > >Regards; >Ahmad > > > >________________________________ >From: Steve Cohen <[email protected]> >To: [email protected] >Sent: Tue, October 5, 2010 4:40:22 PM >Subject: Re: need a larger map task number > >For nutch, I found that updating the values in hadoop-site.xml was enough, >though I also set values for mapred.tasktracker.map.tasks.maximum and >mapred.tasktracker.reduce.tasks.maximum. > >On Tue, Oct 5, 2010 at 9:24 AM, Dennis <[email protected]> wrote: > >> Hi, all >> My "fetch" job uses only 2 map tasks and 2 reduce tasks although I >> configured "mapred.map.tasks" and "mapred.reduce.tasks" in "mapreduce.xml" >> to "32", while I need it run faster.How can I make nutch to use more map and >> reduce tasks when it's fetching? >> Dennis >> >> >> > > > >
