Lewis, I applied the patch you told me. I replaced schema.xml of sol4 installation with schme-sol4.xml. Solr 4.0 system is up and running and I can see its web page with http://localhost:8080/sol40.
I followed tutorial blindly. Crawling went fine but it seem very slow compared to previous before patch applied bin/nutch crawl urls/ -depth 2 But when I index on solr I am getting error. eakarsu@ubuntu:~/searchProject/apache-nutch-2.1/runtime/local$ bin/nutch solrindex http://localhost:8080/sol40/ -reindex SolrIndexerJob: starting Adding 16 documents SolrIndexerJob: java.lang.RuntimeException: job failed: name=solr-index, jobid=job_local_0001 at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:54) at org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:46) at org.apache.nutch.indexer.solr.SolrIndexerJob.indexSolr(SolrIndexerJob.java:54) at org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:75) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.nutch.indexer.solr.SolrIndexerJob.main(SolrIndexerJob.java:84) Unfortunately, tomcat logs does not say anything and does nog log any error. Nov 13, 2012 8:55:41 AM org.apache.solr.core.SolrCore execute INFO: [collection1] webapp=/solr40 path=/admin/system params={wt=json} status=0 QTime=20 Erol Akarsu On Tue, Nov 13, 2012 at 8:35 AM, Lewis John Mcgibbney < [email protected]> wrote: > Hi, > > If you look at the attachments on the issue you will see the patches > for trunk and 2.x which *should* get a pretty comprehensive Nutch + > Solr 4.X stack up and running. Markus made some additional suggestions > which I have unfortunately not had time to integrate into the proposed > fix however your experience on the issue would be excellent. > > Best > > Lewis > > On Tue, Nov 13, 2012 at 1:13 PM, Erol Akarsu <[email protected]> wrote: > > Lewis, > > > > Have you checked it to SVN? Where will I get this patch? > > > > Erol Akarsu > > > > On Tue, Nov 13, 2012 at 6:57 AM, Lewis John Mcgibbney < > > [email protected]> wrote: > > > >> Additionally, please see this issue below and if you are able please > >> provide feedback based on the patch. > >> > >> https://issues.apache.org/jira/browse/NUTCH-1486 > >> > >> hth > >> > >> Lewis > >> > >> On Tue, Nov 13, 2012 at 8:57 AM, Ferdy Galema <[email protected] > > > >> wrote: > >> > I'm not a regular Solr user, but here are some pointers: Somehow, you > >> have > >> > added multiple values for the 'id' field. What did you change from the > >> > default indexing behaviour?Perhaps some custom IndexingFilters? What > >> schema > >> > are you using? (Or as a last resort, perhaps you could give > >> Elasticsearch a > >> > try. This one works out of the box, schema-less). > >> > > >> > On Mon, Nov 12, 2012 at 7:29 PM, Erol Akarsu <[email protected]> > wrote: > >> > > >> >> I am sorry I missed the version of Nutch and SOLR I used to get the > >> above > >> >> error. > >> >> > >> >> I used SOLR 4.0 and Nutch 2.1 > >> >> > >> >> I am hoping somebody would help me to resolved this issue > >> >> > >> >> Erol Akarsu > >> >> > >> >> On Mon, Nov 12, 2012 at 1:02 PM, Erol Akarsu <[email protected]> > wrote: > >> >> > >> >> > I am trying to crawl with Nutch and index on SOLR. Crawling went > fine > >> >> > > >> >> > But when I try to index with SOLR, then I am getting error in my > >> tomcat > >> >> > log file "SEVERE: org.apache.solr.common.SolrException: Document > >> contains > >> >> > multiple values for uniqueKey field: > >> >> > id=[fi.foofactory.blog:http/2007/03/twice-speed-half-size.html, > >> >> > http://blog.foofactory.fi/2007/03/twice-speed-half-size.html, > >> >> > 0000000000000000]" > >> >> > > >> >> > bin/nutch crawl urls/ -depth 2 > >> >> > > >> >> > eakarsu@ubuntu:~/apache-nutch-2.1/runtime/local$ bin/nutch > solrindex > >> >> > http://localhost:8983/solr40/ -reindex > >> >> > SolrIndexerJob: starting > >> >> > Adding 31 documents > >> >> > SolrIndexerJob: java.lang.RuntimeException: job failed: > >> name=solr-index, > >> >> > jobid=job_local_0001 > >> >> > at > org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:54) > >> >> > at > >> >> > > >> org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:46) > >> >> > at > >> >> > > >> >> > >> > org.apache.nutch.indexer.solr.SolrIndexerJob.indexSolr(SolrIndexerJob.java:54) > >> >> > at > >> >> > > >> org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:75) > >> >> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > >> >> > at > >> >> > > >> > org.apache.nutch.indexer.solr.SolrIndexerJob.main(SolrIndexerJob.java:84) > >> >> > > >> >> > > >> >> > Nov 12, 2012 10:42:59 AM org.apache.solr.common.SolrException log > >> >> > SEVERE: org.apache.solr.common.SolrException: Document contains > >> multiple > >> >> > values for uniqueKey field: > >> >> > id=[fi.foofactory.blog:http/2007/03/twice-speed-half-size.html, > >> >> > http://blog.foofactory.fi/2007/03/twice-speed-half-size.html, > >> >> > 0000000000000000] > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.update.AddUpdateCommand.getIndexedId(AddUpdateCommand.java:91) > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:445) > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:325) > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100) > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.update.processor.SignatureUpdateProcessorFactory$SignatureUpdateProcessor.processAdd(SignatureUpdateProcessorFactory.java:181) > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:230) > >> >> > at > >> >> > org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:157) > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92) > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) > >> >> > at > org.apache.solr.core.SolrCore.execute(SolrCore.java:1699) > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455) > >> >> > at > >> >> > > >> >> > >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276) > >> >> > at > >> >> > > >> >> > >> > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > >> >> > at > >> >> > > >> >> > >> > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > >> >> > at > >> >> > > >> >> > >> > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > >> >> > at > >> >> > > >> >> > >> > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > >> >> > at > >> >> > > >> >> > >> > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > >> >> > at > >> >> > > >> >> > >> > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) > >> >> > at > >> >> > > >> >> > >> > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > >> >> > at > >> >> > > >> >> > >> > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) > >> >> > at > >> >> > > >> >> > >> > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) > >> >> > at > >> >> > > >> >> > >> > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602) > >> >> > at > >> >> > > >> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) > >> >> > at java.lang.Thread.run(Thread.java:662) > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > >> > >> > >> > >> -- > >> Lewis > >> > > > > -- > Lewis >
