Markus, You were right. My core was setup properly, however, it was labeled something different in the conf file. I was able to get rid of that error. Thanks!
I have provided the log you asked for below... Dan > -----Original Message----- > From: Markus Jelsma [mailto:[email protected]] > Sent: September 7, 2012 9:49 AM > To: [email protected] > Subject: RE: Errors when indexing to Solr > > -----Original message----- > > From:Fournier, Danny G <[email protected]> > > Sent: Fri 07-Sep-2012 14:46 > > To: [email protected] > > Subject: RE: Errors when indexing to Solr > > > > I've tried crawling with nutch-1.6-SNAPSHOT.jar and got the following > > error: > > > > [root@w7sp1-x64 nutch]# bin/nutch crawl urls -dir crawl -depth 3 -topN 5 > > solrUrl is not set, indexing will be skipped... > > crawl started in: crawl > > rootUrlDir = urls > > threads = 10 > > depth = 3 > > solrUrl=null > > topN = 5 > > Injector: starting at 2012-09-07 08:41:06 > > Injector: crawlDb: crawl/crawldb > > Injector: urlDir: urls > > Injector: Converting injected urls to crawl db entries. > > Injector: Merging injected urls into crawl db. > > Injector: finished at 2012-09-07 08:41:21, elapsed: 00:00:14 > > Generator: starting at 2012-09-07 08:41:21 > > Generator: Selecting best-scoring urls due for fetch. > > Generator: filtering: true > > Generator: normalizing: true > > Generator: topN: 5 > > Generator: jobtracker is 'local', generating exactly one partition. > > Generator: Partitioning selected urls for politeness. > > Generator: segment: crawl/segments/20120907084129 > > Generator: finished at 2012-09-07 08:41:36, elapsed: 00:00:15 > > Fetcher: Your 'http.agent.name' value should be listed first in > > 'http.robots.agents' property. > > Fetcher: starting at 2012-09-07 08:41:36 > > Fetcher: segment: crawl/segments/20120907084129 > > Using queue mode : byHost > > Fetcher: threads: 10 > > Fetcher: time-out divisor: 2 > > QueueFeeder finished: total 1 records + hit by time limit :0 > > Exception in thread "main" java.io.IOException: Job failed! > > at > > org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1265) > > at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:1332) > > at org.apache.nutch.crawl.Crawl.run(Crawl.java:136) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > > at org.apache.nutch.crawl.Crawl.main(Crawl.java:55) > > Please post the relevant log > 2012-09-07 09:54:31,418 WARN mapred.LocalJobRunner - job_local_0005 java.lang.NoClassDefFoundError: com/google/common/util/concurrent/ThreadFactoryBuilder at org.apache.nutch.parse.ParseUtil.<init>(ParseUtil.java:59) at org.apache.nutch.fetcher.Fetcher$FetcherThread.<init>(Fetcher.java:602) at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:1186) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.ThreadFactoryBuild$ at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) ... 6 more
