Hi, Increase your memory in the task trackers by setting your Xmx in mapred.map.child.java.opts.
Cheers -----Original message----- > From:Sznajder ForMailingList <[email protected]> > Sent: Tuesday 2nd July 2013 15:25 > To: [email protected] > Subject: Distributed mode and java/lang/OutOfMemoryError > > Hi, > > I am running Nutch 1.7 on a cluster of 6 nodes. > > I tempted to launch the bin/crawl script in this configuration and I am > getting a very strange error (an error I did not get in the local mode): > > 13/07/02 16:04:23 INFO fetcher.Fetcher: Fetcher Timelimit set for : > 1372781063368 > 13/07/02 16:04:25 INFO mapred.FileInputFormat: Total input paths to process > : 1 > 13/07/02 16:04:27 INFO mapred.JobClient: Running job: job_201207110955_1173 > 13/07/02 16:04:28 INFO mapred.JobClient: map 0% reduce 0% > 13/07/02 16:04:46 INFO mapred.JobClient: map 100% reduce 0% > 13/07/02 16:05:01 INFO mapred.JobClient: Task Id : > attempt_201207110955_1173_r_000000_0, Status : FAILED > Error: null > attempt_201207110955_1173_r_000000_0: JVMDUMP006I Processing dump event > "systhrow", detail "java/lang/OutOfMemoryError" - please wait. > attempt_201207110955_1173_r_000000_0: JVMDUMP032I JVM requested Heap dump > using > '/local/hadoop/tmp/mapred/local/taskTracker/jobcache/job_201207110955_1173/attempt_201207110955_1173_r_000000_0/work/heapdump.20130702.160821.24995.0001.phd' > in response to an event > > > Do you have an idea how to solve that? > > Benjamin >
