[
https://issues.apache.org/jira/browse/NUTCH-49?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883488#action_12883488
]
Alex McLintock commented on NUTCH-49:
-------------------------------------
Is there still interest in this? I'd like to produce a new patch for this if
anyone wants it. It sounds like a good idea to me.
> Flag for generate to fetch only new pages to complement the -refetchonly flag
> -----------------------------------------------------------------------------
>
> Key: NUTCH-49
> URL: https://issues.apache.org/jira/browse/NUTCH-49
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher
> Reporter: Luke Baker
> Priority: Minor
> Attachments: fetchnewonly.patch
>
>
> It would be useful, especially for research/testing purposes, to have a flag
> for the FetchListTool that make sure to only include URLs in the fetchlist
> that have not already been fetched (according to the information from the
> webdb that you're generating the fetchlist from).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
