Hi, > Hi Markus, > > Can you list the steps you executed prior to the solrdedup please?
Not much. It's a crawl running for a few months now. Not all crawls show this problem. > > I think I encountered something similar a while back and as my work was > moving on I didn't get a chance to investigate it fully. > > On Tue, Jun 21, 2011 at 1:54 PM, Markus Jelsma > > <[email protected]>wrote: > > Hi, > > > > Any idea what the exception below can result from? The dedup queries go > > allright and produce normal results. Some indices will not generate this > > NPE. > > > > Cheers, > > > > 11/06/21 20:47:37 WARN mapred.LocalJobRunner: job_local_0001 > > java.lang.NullPointerException > > > > at org.apache.hadoop.io.Text.encode(Text.java:388) > > at org.apache.hadoop.io.Text.set(Text.java:178) > > at > > > > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat$1.next > > (SolrDeleteDuplicates.java:272) > > > > at > > > > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat$1.next > > (SolrDeleteDuplicates.java:243) > > > > at > > > > org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.j > > ava:192) > > > > at > > > > org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:17 > > 6) > > > > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48) > > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > > at > > > > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177) > > 11/06/21 20:47:37 INFO mapred.JobClient: map 0% reduce 0% > > 11/06/21 20:47:37 INFO mapred.JobClient: Job complete: job_local_0001 > > 11/06/21 20:47:37 INFO mapred.JobClient: Counters: 0 > > Exception in thread "main" java.io.IOException: Job failed! > > > > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252) > > at > > > > org.apache.nutch.indexer.solr.SolrDeleteDuplicates.dedup(SolrDeleteDuplic > > ates.java:363) > > > > at > > > > org.apache.nutch.indexer.solr.SolrDeleteDuplicates.run(SolrDeleteDuplicat > > es.java:375) > > > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > > at > > > > org.apache.nutch.indexer.solr.SolrDeleteDuplicates.main(SolrDeleteDuplica > > tes.java:380)
