installed nutch 1.2 and run this:
bin/nutch crawl urls -dir result -depth 3 >& crawl.log
Here is the logs/hadoop.log:
2010-10-06 16:04:40,350 INFO crawl.Crawl - crawl started in: result
2010-10-06 16:04:40,351 INFO crawl.Crawl - rootUrlDir = urls
2010-10-06 16:04:40,351 INFO crawl.Crawl - threads = 10
2010-10-06 16:04:40,351 INFO crawl.Crawl - depth = 3
2010-10-06 16:04:40,351 INFO crawl.Crawl - indexer=lucene
2010-10-06 16:04:40,370 INFO crawl.Injector - Injector: starting at
2010-10-06 16:04:40
2010-10-06 16:04:40,370 INFO crawl.Injector - Injector: crawlDb:
result/crawldb
2010-10-06 16:04:40,370 INFO crawl.Injector - Injector: urlDir: urls
2010-10-06 16:04:40,430 INFO crawl.Injector - Injector: Converting injected
urls to crawl db entries.
2010-10-06 16:04:40,494 WARN mapred.JobClient - Use GenericOptionsParser
for parsing the arguments. Applications should implement Tool for the same.
....
2010-10-06 16:04:42,097 INFO plugin.PluginRepository - Nutch
Content Parser (org.apache.nutch.parse.Parser)
2010-10-06 16:04:42,097 INFO plugin.PluginRepository - Nutch
Scoring (org.apache.nutch.scoring.ScoringFilter)
2010-10-06 16:04:42,098 INFO plugin.PluginRepository - Ontology
Model Loader (org.apache.nutch.ontology.Ontology)
2010-10-06 16:04:42,127 WARN mapred.LocalJobRunner - job_local_0001
java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 13 more
Caused by: java.lang.NoSuchMethodError:
org.apache.log4j.Logger.isTraceEnabled()Z
at
org.slf4j.impl.Log4jLoggerAdapter.isTraceEnabled(Log4jLoggerAdapter.java:81)
at
org.apache.commons.logging.impl.SLF4JLocationAwareLog.isTraceEnabled(SLF4JLocationAwareLog.java:52)
at
org.apache.nutch.urlfilter.api.RegexURLFilterBase.readRulesFile(RegexURLFilterBase.java:191)
at
org.apache.nutch.urlfilter.api.RegexURLFilterBase.setConf(RegexURLFilterBase.java:141)
at
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:162)
at org.apache.nutch.net.URLFilters.<init>(URLFilters.java:57)
at
org.apache.nutch.crawl.Injector$InjectMapper.configure(Injector.java:72)
... 18 more
Any insights?
Thanks in advance.
