the explanation can be found in the stack trace you sent : "java.io.IOException: error=12, Cannot allocate memory"
Small instances on EC2 does not give you enough memory. from the configuration below the slaves will use up to 1300M for the datanode and tasktracker; if you add to that the memory used by the OS and of course the tasks themselves it is not surprising that you used the 1.7G you had. Things get worse if you parse at the same time as you fetch as this tend to take some RAM. >From my experience EC2 large instances are more appropriate for a Nutch cluster PS: nutch-user would be a more appropriate list for this type of messages J. -- DigitalPebble Ltd http://www.digitalpebble.com On 10 February 2010 08:41, Santiago Pérez <elara...@gmail.com> wrote: > > Hej > > I am running Nutch in a cluster with 1 master and 6 slaves in Amazon (with > the same instances for all of them with 1.7GB RAM memory) > > My configuration is the following: > > HADOOP_HEAPSIZE=1300 > HADOOP_NAMENODE_OPTS=-Xmx400m > HADOOP_SECONDARYNAMENODE_OPTS=-Xmx400m > HADOOP_JOBTRACKER_OPTS=-Xmx400m > dfs.replication=3 > mapred.map.tasks=6 > mapred.reduce.tasks=6 > mapred.child.java.opts=-Xmx950m > > But in the second depth fetch, I got the following errors in some instances > (while the other ones seems they fetched correctly) : > > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - java.io.IOException: Spill > failed > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:822) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:466) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.nutch.fetcher.Fetcher$FetcherThread.output(Fetcher.java:907) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:670) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - Caused by: > java.io.IOException: Cannot run program "bash": java.io.IOException: > error=12, Cannot allocate memory > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > java.lang.ProcessBuilder.start(ProcessBuilder.java:459) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.util.Shell.runCommand(Shell.java:149) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.util.Shell.run(Shell.java:134) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.fs.DF.getAvailable(DF.java:73) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:329) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:107) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1183) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$1800(MapTask.java:648) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer$SpillThread.run(MapTask.java:1135) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - Caused by: > java.io.IOException: java.io.IOException: error=12, Cannot allocate memory > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > java.lang.UNIXProcess.<init>(UNIXProcess.java:148) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > java.lang.ProcessImpl.start(ProcessImpl.java:65) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - at > java.lang.ProcessBuilder.start(ProcessBuilder.java:452) > 2010-02-10 03:18:31,185 FATAL fetcher.Fetcher - ... 9 more > . > . > . > . > . > 2010-02-10 03:18:31,463 WARN mapred.TaskTracker - Error running child > java.io.IOException: Spill failed > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1085) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:359) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > at org.apache.hadoop.mapred.Child.main(Child.java:170) > Caused by: java.io.IOException: Cannot run program "bash": > java.io.IOException: error=12, Cannot allocate memory > at java.lang.ProcessBuilder.start(ProcessBuilder.java:459) > at org.apache.hadoop.util.Shell.runCommand(Shell.java:149) > at org.apache.hadoop.util.Shell.run(Shell.java:134) > at org.apache.hadoop.fs.DF.getAvailable(DF.java:73) > at > org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:329) > at > org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124) > at > org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:107) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1183) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.access$1800(MapTask.java:648) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer$SpillThread.run(MapTask.java:1135) > Caused by: java.io.IOException: java.io.IOException: error=12, Cannot > allocate memory > at java.lang.UNIXProcess.<init>(UNIXProcess.java:148) > at java.lang.ProcessImpl.start(ProcessImpl.java:65) > at java.lang.ProcessBuilder.start(ProcessBuilder.java:452) > ... 9 more > > Any idea?? > > Thanks in advance :) > -- > View this message in context: > http://old.nabble.com/Spill-failed-tp27527090p27527090.html > Sent from the Nutch - Dev mailing list archive at Nabble.com. > >
