Hi Susam, I get this error for both cases 1 and 2.
I think it's due to running hadoop in local mode (ie single machine). It seems it's always giving a jobid of 1. I've been using only a single thread so i'm not sure why this is; then again I don't really understand how the whole nutch/hadoop system works ... The weird thing is, sometimes the script (both yours and bin/nutch) will run all the way through, sometimes for 1 or 2 "depths" of a crawl, sometimes for the injecting of urls. It's seemingly random. I've found nothing online to help out. Thanks Susam! On 1/9/08, Susam Pal <[EMAIL PROTECTED]> wrote: > > I haven't really worked with the latest trunk. But I am wondering if ... > > 1. you get this error when you kill a crawl while it is running, i.e. > the unfinished crawl is killed and then start a new crawl > > 2. you get this error when you crawl using 'bin/nutch crawl' command > as well as the crawl script? > > Regards, > Susam Pal > > > Hi there, > > > > I'm having problems running the latest release of nutch. I get the > following > > error when I try to crawl: > > > > Fetcher: segment: crawl/segments/20080109183955 > > Fetcher: java.io.IOException: Target > > /tmp/hadoop-me/mapred/local/localRunner/job_local_1.xml already exists > > at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246) > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:125) > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:116) > > at org.apache.hadoop.fs.LocalFileSystem.copyToLocalFile( > > LocalFileSystem.java:55) > > at org.apache.hadoop.fs.FileSystem.copyToLocalFile( > FileSystem.java > > :834) > > at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>( > > LocalJobRunner.java:86) > > at org.apache.hadoop.mapred.LocalJobRunner.submitJob( > > LocalJobRunner.java:281) > > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java > :558) > > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:753) > > at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:526) > > at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:561) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > > at org.apache.hadoop.util.ToolBase.doMain(ToolBase.java:54) > > at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:533) > > > > If I manually remove the offending directory it works... sometimes. > > > > Any help is appreciated. > > > > Regards, > > IWan > > >
