Hi Manikandan I presume you are running it from runtime/deploy/bin. Have you tried looking into the job jar file in runtime/deploy to make sure the class is in the archive?
Julien On 26 May 2014 19:20, Manikandan Saravanan <[email protected]>wrote: > Hi, > > I’m running Nutch 2 on a Hadoop 1.2.1 cluster with 2 nodes. I’m running > Solr 4 separately on a box and I replaced Solr’s schema with Nutch’s Solr-4 > schema. When I run a crawl, I get the following error at the end of the job > > 14/05/26 14:08:32 INFO solr.SolrDeleteDuplicates: SolrDeleteDuplicates: > starting... > 14/05/26 14:08:32 INFO solr.SolrDeleteDuplicates: SolrDeleteDuplicates: > Solr url: http://10.130.231.16:8983/solr/nutch > 14/05/26 14:08:33 WARN mapred.JobClient: No job jar file set. User > classes may not be found. See JobConf(Class) or JobConf#setJar(String). > 14/05/26 14:08:33 INFO mapred.JobClient: Running job: job_201405261214_0014 > 14/05/26 14:08:34 INFO mapred.JobClient: map 0% reduce 0% > 14/05/26 14:08:43 INFO mapred.JobClient: Task Id : > attempt_201405261214_0014_m_000000_0, Status : FAILED > java.lang.RuntimeException: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857) > at > org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:270) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855) > ... 8 more > > 14/05/26 14:08:43 WARN mapred.JobClient: Error reading task > outputnutch-two-qontifi > 14/05/26 14:08:43 WARN mapred.JobClient: Error reading task > outputnutch-two-qontifi > 14/05/26 14:08:44 INFO mapred.JobClient: Task Id : > attempt_201405261214_0014_m_000001_0, Status : FAILED > java.lang.RuntimeException: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857) > at > org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:270) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855) > ... 8 more > > 14/05/26 14:08:50 INFO mapred.JobClient: Task Id : > attempt_201405261214_0014_m_000001_1, Status : FAILED > java.lang.RuntimeException: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857) > at > org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:270) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855) > ... 8 more > > 14/05/26 14:08:50 WARN mapred.JobClient: Error reading task > outputnutch-two-qontifi > 14/05/26 14:08:50 WARN mapred.JobClient: Error reading task > outputnutch-two-qontifi > 14/05/26 14:08:51 INFO mapred.JobClient: Task Id : > attempt_201405261214_0014_m_000000_1, Status : FAILED > java.lang.RuntimeException: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857) > at > org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:270) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855) > ... 8 more > > 14/05/26 14:08:56 INFO mapred.JobClient: Task Id : > attempt_201405261214_0014_m_000001_2, Status : FAILED > java.lang.RuntimeException: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857) > at > org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:270) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855) > ... 8 more > > 14/05/26 14:08:56 WARN mapred.JobClient: Error reading task > outputnutch-two-qontifi > 14/05/26 14:08:56 WARN mapred.JobClient: Error reading task > outputnutch-two-qontifi > 14/05/26 14:08:57 INFO mapred.JobClient: Task Id : > attempt_201405261214_0014_m_000000_2, Status : FAILED > java.lang.RuntimeException: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857) > at > org.apache.hadoop.mapreduce.JobContext.getInputFormatClass(JobContext.java:187) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.ClassNotFoundException: > org.apache.nutch.indexer.solr.SolrDeleteDuplicates$SolrInputFormat > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:270) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855) > > And finally, nothing is being indexed to Solr. Am I missing anything? > Please help. > > Regards, > Manikandan -- Open Source Solutions for Text Engineering http://digitalpebble.blogspot.com/ http://www.digitalpebble.com http://twitter.com/digitalpebble
