Hi all, I'm now using nutch 0.7.1. I am using whole-web crawling method and i had successfully indexed the segments. I crawled totally 4 website but what I got in the end only 59 pages and 55 links from database. Then I generated another segment and fetched again from same website, after updated my database and read it, i got 328 pages and 451 links. Then third time i even got 879 pages and 1673 links. I wonder why i could only get 50 plus pages and links fetched at first time while hundreds or thousands of them at following times? is it strange my result like this or it is usual? Before it, I had changed some of the property in both nutch-default.xml and nutch-site.xml, changed property I had listed below : http.time.out 1000000 http.content.limit -1 http.max.delays 5 fetcher.server.delay 20 Thank you all very much for your attentiion to my problems.
Send instant messages to your online friends http://uk.messenger.yahoo.com
