Sami Siren wrote:
Sami Siren wrote:
I can see this error also. not sure yet what's going wrong...
it's NUTCH-703 (hadoop upgrade) that broke the indexing. any ideas
what changed in hadoop that might have caused this?
found the hostile hadoop commit:
http://svn.apache.org/viewvc?view=rev&revision=736239
any ideas how to proceed? Naturally i won't be starting the releace
proces before this is resolved.
--
Sami Siren
--
Sami Siren
--
Sami Siren
Justin Yao wrote:
log4j configure:
log4j.logger.org.apache.nutch.indexer.Indexer=TRACE,cmdstdout
log4j.logger.org.apache.nutch=TRACE
log4j.logger.org.apache.hadoop=TRACE
Output:
2009-03-02 17:53:21,987 DEBUG indexer.Indexer - IFD [Thread-11]:
setInfoStream
deletionpolicy=org.apache.lucene.index.keeponlylastcommitdeletionpol...@118d189
2009-03-02 17:53:21,988 DEBUG indexer.Indexer - IW 0 [Thread-11]:
setInfoStream:
dir=org.apache.lucene.store.FSDirectory@/tmp/hadoop-justin/mapred/local/index/_1068960877
autoCommit=true
mergepolicy=org.apache.lucene.index.logbytesizemergepol...@648016
mergescheduler=org.apache.lucene.index.concurrentmergeschedu...@1551b0
ramBufferSizeMB=16.0 maxBufferedDocs=50 maxBuffereDeleteTerms=-1
maxFieldLength=10000 index=
2009-03-02 17:53:21,993 INFO indexer.IndexingFilters - Adding
org.apache.nutch.indexer.basic.BasicIndexingFilter
2009-03-02 17:53:21,994 INFO indexer.IndexingFilters - Adding
org.apache.nutch.indexer.anchor.AnchorIndexingFilter
2009-03-02 17:53:22,009 WARN mapred.LocalJobRunner - job_local_0001
java.lang.IllegalArgumentException: it doesn't make sense to have a
field that is neither indexed nor stored
at org.apache.lucene.document.Field.<init>(Field.java:279)
at
org.apache.nutch.indexer.lucene.LuceneWriter.createLuceneDoc(LuceneWriter.java:133)
at
org.apache.nutch.indexer.lucene.LuceneWriter.write(LuceneWriter.java:239)
at
org.apache.nutch.indexer.IndexerOutputFormat$1.write(IndexerOutputFormat.java:50)
at
org.apache.nutch.indexer.IndexerOutputFormat$1.write(IndexerOutputFormat.java:40)
at
org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:410)
at
org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:158)
at
org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:50)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:436)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:170)
2009-03-02 17:53:22,567 FATAL indexer.Indexer - Indexer:
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232)
at org.apache.nutch.indexer.Indexer.index(Indexer.java:72)
at org.apache.nutch.indexer.Indexer.run(Indexer.java:92)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.indexer.Indexer.main(Indexer.java:101)
Andrzej Bialecki wrote:
Justin Yao wrote:
Same problem here if using build #740 (Mar 2, 2009 4:01:53 AM)
I switched to build #736 (Feb 26, 2009 4:01:15 AM) and it worked
then.
Could you please send the error message from the logs/, which you
got with build #740? Thanks!
