I added the property to hadoop-site.xml, and it now fails with a different
error. How about some more magical properties?
Thanks.
Phillip
<property>
<name>mapred.speculative.execution</name>
<value>false</value>
<description>from the mailing list
</description>
</property>
Stack trace:
2006-12-22 13:40:46,040 WARN mapred.LocalJobRunner
(LocalJobRunner.java:run(147)) - job_ambth9
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.lucene.index.MultiReader.isDeleted(MultiReader.java:109)
at
org.apache.nutch.indexer.DeleteDuplicates$InputFormat$DDRecordReader.next(DeleteDuplicates.java:177)
at org.apache.hadoop.mapred.MapTask$3.next(MapTask.java:203)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:46)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:215)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:109)
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:399)
at
org.apache.nutch.indexer.DeleteDuplicates.dedup(DeleteDuplicates.java:433)
at org.apache.nutch.crawl.Crawl.main(Crawl.java:131)
----- Original Message -----
From: "Andrzej Bialecki" <[EMAIL PROTECTED]>
To: <[email protected]>
Sent: Friday, December 22, 2006 12:50 PM
Subject: Re: PhasedFileSystem Exception in trunk build
spamsucks wrote:
I pulled from trunk yesterday, and doing a Intranet crawl, it fails.
Running on cygwin/XP
Any guidance? Thanks.
2006-12-22 11:10:00,613 INFO indexer.IndexingFilters - Adding
org.apache.nutch.indexer.basic.BasicIndexingFilter
2006-12-22 11:10:01,363 WARN mapred.LocalJobRunner - job_g7zf6p
java.lang.UnsupportedOperationException: Operation not supported
at
org.apache.hadoop.mapred.PhasedFileSystem.deleteRaw(PhasedFileSystem.java:291)
at org.apache.hadoop.fs.FileSystem.delete(FileSystem.java:402)
at
org.apache.nutch.indexer.Indexer$OutputFormat.getRecordWriter(Indexer.java:97)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:287)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:137)
Please set "mapred.speculative.execution" to false, in hadoop-site.xml.
This is addressed in the upcoming update of Hadoop v. 0.9.2.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
