Right then.. compiled the svn version of nutch. Tried running the
crawl with it and this is the log:
server: 11:32pm % ./bin/nutch crawl ../SpectraSearch/urls -dir
../SpectraSearch/crawl -depth 2 -threads 20
060305 233255 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/hadoop-default.xml
060305 233255 parsing file:/home/hdiwan/nutch/conf/nutch-default.xml
060305 233255 parsing file:/home/hdiwan/nutch/conf/crawl-tool.xml
060305 233255 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/mapred-default.xml
060305 233255 parsing file:/home/hdiwan/nutch/conf/nutch-site.xml
060305 233255 parsing file:/home/hdiwan/nutch/conf/hadoop-site.xml
060305 233256 crawl started in: ../SpectraSearch/crawl
060305 233256 rootUrlDir = ../SpectraSearch/urls
060305 233256 threads = 20
060305 233256 depth = 2
060305 233256 Injector: starting
060305 233256 Injector: crawlDb: ../SpectraSearch/crawl/crawldb
060305 233256 Injector: urlDir: ../SpectraSearch/urls
060305 233256 Injector: Converting injected urls to crawl db entries.
060305 233256 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/hadoop-default.xml
060305 233256 parsing file:/home/hdiwan/nutch/conf/nutch-default.xml
060305 233256 parsing file:/home/hdiwan/nutch/conf/crawl-tool.xml
060305 233256 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/mapred-default.xml
060305 233256 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/mapred-default.xml
060305 233256 parsing file:/home/hdiwan/nutch/conf/nutch-site.xml
060305 233256 parsing file:/home/hdiwan/nutch/conf/hadoop-site.xml
060305 233256 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/hadoop-default.xml
060305 233256 parsing file:/home/hdiwan/nutch/conf/nutch-default.xml
060305 233256 parsing file:/home/hdiwan/nutch/conf/crawl-tool.xml
060305 233256 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/mapred-default.xml
060305 233256 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/mapred-default.xml
060305 233256 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/mapred-default.xml
060305 233256 parsing file:/home/hdiwan/nutch/conf/nutch-site.xml
060305 233256 parsing file:/home/hdiwan/nutch/conf/hadoop-site.xml
060305 233256 Running job: job_7n6bsm
060305 233256 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/hadoop-default.xml
060305 233256 parsing
jar:file:/home/hdiwan/nutch/lib/hadoop-0.1-dev.jar!/mapred-default.xml
060305 233256 parsing /tmp/hadoop/mapred/local/localRunner/job_7n6bsm.xml
060305 233256 parsing file:/home/hdiwan/nutch/conf/hadoop-site.xml
java.io.IOException: No input directories specified in: Configuration:
defaults: hadoop-default.xml , mapred-default.xml ,
/tmp/hadoop/mapred/local/localRunner/job_7n6bsm.xmlfinal:
hadoop-site.xml
at
org.apache.hadoop.mapred.InputFormatBase.listFiles(InputFormatBase.java:84)
at
org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:94)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:70)
060305 233257 map 0% reduce 0%
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:310)
at org.apache.nutch.crawl.Injector.inject(Injector.java:114)
at org.apache.nutch.crawl.Crawl.main(Crawl.java:104)
I need to sleep now, so I'll check back tomorrow. Thanks for all the help!
--
Cheers,
Hasan Diwan <[EMAIL PROTECTED]>
N�HS^�隊X���'���u��<�ڂ�.���y�"� �*m�x%jx.j���^�קvƩ�X�jب�ȧ�� m�ݚ�����v&��קv�^�+����j�Z����{az����^��h���n���)��{h���� �ا��+h�(m�����Z�� jY�w��ǥrg
