Hi, I'm using nutch 2.x with HBase and tried to crawl " http://www.halliburton.com/en-US/default.page"; site for depth level 5.
Following is the command: bin/crawl urls/seed.txt HB http://localhost:8080/solr/ 5 It worked well till 3rd iteration but for remaining 4th and 5th nothing fetched (same case happened with cnn.com). but if i tried to crawl other sites like amazon with depth level 5 it works. Could you please guide what could be the reasons for failing of 4th and 5th iteration. Regards, Jamshaid
