I'm assuming that if your running on an established hadoop cluster you will wish to keep it over there. Lewis
On Tuesday, July 2, 2013, Sznajder ForMailingList <[email protected]> wrote: > Thanks > Can I copy this file to my $NUTCH/conf directory, or must I keep it in the > $HADOOP/conf directory? > > Benjamin > > > On Tue, Jul 2, 2013 at 5:10 PM, Lewis John Mcgibbney < > [email protected]> wrote: > >> in mapred-site.xml >> It is your Mapreduce configuration override. >> hth >> >> On Tuesday, July 2, 2013, Sznajder ForMailingList < >> [email protected]> >> wrote: >> > Thanks a lot Markus! >> > >> > Where do we define this parameter, please? >> > >> > Benjamin >> > >> > >> > On Tue, Jul 2, 2013 at 4:28 PM, Markus Jelsma < >> [email protected] >> >wrote: >> > >> >> Hi, >> >> >> >> Increase your memory in the task trackers by setting your Xmx in >> >> mapred.map.child.java.opts. >> >> >> >> Cheers >> >> >> >> >> >> >> >> -----Original message----- >> >> > From:Sznajder ForMailingList <[email protected]> >> >> > Sent: Tuesday 2nd July 2013 15:25 >> >> > To: [email protected] >> >> > Subject: Distributed mode and java/lang/OutOfMemoryError >> >> > >> >> > Hi, >> >> > >> >> > I am running Nutch 1.7 on a cluster of 6 nodes. >> >> > >> >> > I tempted to launch the bin/crawl script in this configuration and I >> am >> >> > getting a very strange error (an error I did not get in the local >> mode): >> >> > >> >> > 13/07/02 16:04:23 INFO fetcher.Fetcher: Fetcher Timelimit set for : >> >> > 1372781063368 >> >> > 13/07/02 16:04:25 INFO mapred.FileInputFormat: Total input paths to >> >> process >> >> > : 1 >> >> > 13/07/02 16:04:27 INFO mapred.JobClient: Running job: >> >> job_201207110955_1173 >> >> > 13/07/02 16:04:28 INFO mapred.JobClient: map 0% reduce 0% >> >> > 13/07/02 16:04:46 INFO mapred.JobClient: map 100% reduce 0% >> >> > 13/07/02 16:05:01 INFO mapred.JobClient: Task Id : >> >> > attempt_201207110955_1173_r_000000_0, Status : FAILED >> >> > Error: null >> >> > attempt_201207110955_1173_r_000000_0: JVMDUMP006I Processing dump >> event >> >> > "systhrow", detail "java/lang/OutOfMemoryError" - please wait. >> >> > attempt_201207110955_1173_r_000000_0: JVMDUMP032I JVM requested Heap >> dump >> >> > using >> >> > >> >> >> >> '/local/hadoop/tmp/mapred/local/taskTracker/jobcache/job_201207110955_1173/attempt_201207110955_1173_r_000000_0/work/heapdump.20130702.160821.24995.0001.phd' >> >> > in response to an event >> >> > >> >> > >> >> > Do you have an idea how to solve that? >> >> > >> >> > Benjamin >> >> > >> >> >> > >> >> -- >> *Lewis* >> > -- *Lewis*
