NM. Looks like it's an issue with running multiple jobs at once. http://mail-archives.apache.org/mod_mbox/nutch-user/201009.mbox/%[email protected]%3E
On Fri, Oct 28, 2011 at 11:18 AM, Bai Shen <[email protected]> wrote: > I'm using hadoop through nutch 1.4 I'm using the native libraries, but > this happens without them as well. I'm only using one local node. > Sometimes when running a nutch fetch job, I get the following error. There > doesn't seem to be any rhyme or reason to it AFAIK. > > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find > taskTracker/jobcache/job_local_0001/attempt_local_0001_m_000000_0/output/spill0.out > in any of the configured local directories > > at > org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389) > > at > org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138) > > at > org.apache.hadoop.mapred.MapOutputFile.getSpillFile(MapOutputFile.java:94) > > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1443) > > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1154) > > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:359) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) > > 2011-10-28 10:49:44,415 ERROR fetcher.Fetcher - Fetcher: > java.io.IOException: Job failed! > > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252) > > at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:1202) > > at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:1238) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > > at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:1211) >
