Hi, Is Nutch 2.x capable of issuing a GET request, comparing the reported Last-Modified date with the last modified date from the previous fetch of a page and, if the page is deemed unmodified since the last fetch, avoid fetching the rest of the page?
.... and thus save bandwidth (and maybe speed up fetching)? Thanks, Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/
