I am using Nutch 2.1 every time I run crawl from dmoz directory my existing crawled pages in the database are fetched again(Taking long time/). Is there a way to crawl only new sites.
Thank you -- Jayadeep Reddy.S, M.D & C.E.O e Health Access Pvt.Ltd www.ehealthaccess.com Hyderabad-Chennai-Banglore http://www.youtube.com/watch?v=0k5LX8mw6Sk
