greeting, I wanna know how to make cralwer fetch pages as much as it can? I want to downsize the unfetched pages number.
In the first time, I used default value 100 for db.max.outlinks.per.page. I noticed there are lots of unfetched url compare to fetched url.. about 15000 against 3000 Then, I tried to set db.max.outlinks.per.page to -1, but the rate is similar to 15000 vs 3000. I also tried set it to 0, nothing is fetched. Thanks Ian -- View this message in context: http://www.nabble.com/about-unfetched-links-tp20851584p20851584.html Sent from the Nutch - User mailing list archive at Nabble.com.
