Hello, I'm using nutch 2.1 with mysql and when I do a simple "bin/nutch crawl seed/ -depth 5 -topN 10000", I noticed nutch fetch 3 or 4 times the same URL during the crawl, why ?
I just configured nutch to local crawl a website (restriction in regex-urlfilter), everything else looks ok on mysql. nuch-site.xml : http://pastebin.com/Mx9s5Kfz
