Nutch 0.9 Anyone know if it is possible to be more granular regarding crawl frequency? Meaning, that I would like some sites to be crawled more often then others. Like, a news site should be crawled every day, but your average business website should be crawled every 30 days. So, is it possible to specify a crawl frequency for specific urls, or is it only global for within the crawl db? I suppose I could have several crawldbs or something like that, and deal with it.. but, just curious.
Thanks -- "Conscious decisions by conscious minds are what make reality real"
