sure, I realize its just the indexing backends plugin that's changed am about to run a full set again but the only two I had problems with where de-duplication and indexing
more soon ps: my next question will be how to script this, those Hadoop command lines are doing my head in On Wed, Aug 14, 2013 at 12:48 PM, Markus Jelsma <[email protected]>wrote: > Also, the webgraph is not part of indexing. That just has a ScoreUpdater > tool that writes scores back to the crawldb, those are still passed via the > boost field in IndexerMapReduce. > > -----Original message----- > > From:Nicholas Roberts <[email protected]> > > Sent: Wednesday 14th August 2013 21:44 > > To: [email protected] > > Subject: Re: Nutch 1.7 on Hadoop Exception in thread "main" > java.lang.ClassNotFoundException: org.apache.nutch.indexer.solr.SolrIndexer > > > > ok, cracked open the src and found IndexingJob and this below works > > > > however, I read in that JIRA issue that there would be backwards > > compatability? Webgraph, Linkdb etc all work as before, so is it hard to > be > > backwards compatible? > > > > sudo -u hdfs hadoop jar > > /opt/nutch/apache-nutch-1.7/build/apache-nutch-1.7.job > > org.apache.nutch.indexer.IndexingJob -D solr.server.url= > > http://solr.server.tld:8088/solr/core1/ /user/crawl-1.7-10-5000/crawldb > > -linkdb /user/crawl-1.7-10-5000/linkdb -dir > /user/crawl-1.7-10-5000/segments > > > > > > > > On Wed, Aug 14, 2013 at 12:21 PM, Nicholas Roberts < > > [email protected]> wrote: > > > > > I read that previously, but I wasn't sure exactly how I was to run a > > > Hadoop job > > > > > > so, the old Hadoop methods are no longer supported? > > > > > > is there an equivalent to below in the new indexer backend ? > > > > > > sudo -u hdfs hadoop jar > > > > /opt/nutch/apache-nutch-1.7/build/apache-nutch-1.7.job > > > > org.apache.nutch.indexer.solr.SolrIndexer -solr > > > > http://solr.server.tld:8088/solr/core1/ /user/crawl-1.7-1/crawldb > > > -linkdb > > > > /user/crawl-1.7-1/linkdb -dir /user/crawl-1.7-1/segments > > > > > > > > > On Wed, Aug 14, 2013 at 11:33 AM, Markus Jelsma < > > > [email protected]> wrote: > > > > > >> That's right. Check NUTCH-1047, that is what changed: > > >> https://issues.apache.org/jira/browse/NUTCH-1047 > > >> > > >> -----Original message----- > > >> > From:Nicholas Roberts <[email protected]> > > >> > Sent: Wednesday 14th August 2013 20:11 > > >> > To: [email protected] > > >> > Subject: Nutch 1.7 on Hadoop Exception in thread "main" > > >> java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrIndexer > > >> > > > >> > hi > > >> > > > >> > I am testing upgrading from Nutch 1.6 to Nutch 1.7 and seem to have > a > > >> > problem with the SolrIndexer > > >> > > > >> > on Nutch 1.6 this works fine > > >> > > > >> > sudo -u hdfs hadoop jar > > >> > /opt/nutch/apache-nutch-1.7/build/apache-nutch-1.7.job > > >> > org.apache.nutch.indexer.solr.SolrIndexer -solr > > >> > http://solr.server.tld:8088/solr/core1/ /user/crawl-1.7-1/crawldb > > >> -linkdb > > >> > /user/crawl-1.7-1/linkdb -dir /user/crawl-1.7-1/segments > > >> > > > >> > > > >> > on Nutch 1.7 I get error > > >> > > > >> > > > >> > Exception in thread "main" java.lang.ClassNotFoundException: > > >> > org.apache.nutch.indexer.solr.SolrIndexer > > >> > > > >> > > > >> > Exception in thread "main" java.lang.ClassNotFoundException: > > >> > org.apache.nutch.indexer.solr.SolrIndexer > > >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > > >> > at java.security.AccessController.doPrivileged(Native > Method) > > >> > at > java.net.URLClassLoader.findClass(URLClassLoader.java:190) > > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) > > >> > at java.lang.Class.forName0(Native Method) > > >> > at java.lang.Class.forName(Class.java:247) > > >> > at org.apache.hadoop.util.RunJar.main(RunJar.java:201) > > >> > > > >> > -- > > >> > > > >> > -- > > >> > Nicholas Roberts > > >> > US 510-684-8264 > > >> > http://Permaculture.TV <http://permaculture.tv/> > > >> > > > >> > > > > > > > > > > > > -- > > > > > > -- > > > Nicholas Roberts > > > US 510-684-8264 > > > http://Permaculture.TV <http://permaculture.tv/> > > > > > > > > > > > -- > > > > -- > > Nicholas Roberts > > US 510-684-8264 > > http://Permaculture.TV <http://permaculture.tv/> > > > -- -- Nicholas Roberts US 510-684-8264 http://Permaculture.TV <http://permaculture.tv/>
