I added the property to hadoop-site.xml, and it now fails with a different
error. How about some more magical properties?
Thanks.
Phillip
<property>
<name>mapred.speculative.execution</name>
<value>false</value>
<description>from the mailing list
</description>
</property>
Stack trace:
2006-12-22 13:40:46,040 WARN mapred.LocalJobRunner
(LocalJobRunner.java:run(147)) - job_ambth9
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.lucene.index.MultiReader.isDeleted(MultiReader.java:109)
at
org.apache.nutch.indexer.DeleteDuplicates$InputFormat$DDRecordReader.next(DeleteDuplicates.java:177)
at org.apache.hadoop.mapred.MapTask$3.next(MapTask.java:203)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:46)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:215)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:109)
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:399)
at
org.apache.nutch.indexer.DeleteDuplicates.dedup(DeleteDuplicates.java:433)
at org.apache.nutch.crawl.Crawl.main(Crawl.java:131)
----- Original Message -----
From: "Andrzej Bialecki" <[EMAIL PROTECTED]>
To: <[email protected]>
Sent: Friday, December 22, 2006 12:50 PM
Subject: Re: PhasedFileSystem Exception in trunk build
> spamsucks wrote:
>> I pulled from trunk yesterday, and doing a Intranet crawl, it fails.
>>
>> Running on cygwin/XP
>>
>> Any guidance? Thanks.
>>
>> 2006-12-22 11:10:00,613 INFO indexer.IndexingFilters - Adding
>> org.apache.nutch.indexer.basic.BasicIndexingFilter
>> 2006-12-22 11:10:01,363 WARN mapred.LocalJobRunner - job_g7zf6p
>> java.lang.UnsupportedOperationException: Operation not supported
>> at
>> org.apache.hadoop.mapred.PhasedFileSystem.deleteRaw(PhasedFileSystem.java:291)
>> at org.apache.hadoop.fs.FileSystem.delete(FileSystem.java:402)
>> at
>> org.apache.nutch.indexer.Indexer$OutputFormat.getRecordWriter(Indexer.java:97)
>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:287)
>> at
>> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:137)
>
> Please set "mapred.speculative.execution" to false, in hadoop-site.xml.
> This is addressed in the upcoming update of Hadoop v. 0.9.2.
>
> --
> Best regards,
> Andrzej Bialecki <><
> ___. ___ ___ ___ _ _ __________________________________
> [__ || __|__/|__||\/| Information Retrieval, Semantic Web
> ___|||__|| \| || | Embedded Unix, System Integration
> http://www.sigram.com Contact: info at sigram dot com
>
>
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
