Stopping at depth=0 - no more URLs to fetch

2009-11-11 Thread kvorion
Hi all, I have been trying to run a crawl on a couple of different domains using nutch: bin/nutch crawl urls -dir crawled -depth 3 Everytime I get the response: Stopping at depth=x - no more URLs to fetch. Sometimes a page or two at the first level get crawled and in most other cases, nothing

nutch search yields 0 results

2009-11-10 Thread kvorion
Hi all... I was finally able to set up a multinode nutch cluster that seemed to work fine. When I set it up to do the example crawl of http://lucene.apache.org then the crawl seemed to finish successfully as indicated by the output on the console. When I copied the index files on to the local fil

Cannot get slave nodes to run

2009-11-09 Thread kvorion
Hi All... I have been trying to set up nutch on a cluster of 3 machines. I could get the crawling and searching process to run independently on all 3 machines but when I try to integrate them as a single cluster, then none of the slaves are shown in the listing of nodes on the Hadoop Machine List