[
https://issues.apache.org/jira/browse/NUTCH-49?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883492#action_12883492
]
Julien Nioche commented on NUTCH-49:
------------------------------------
Can't you do the same by implementing a custom FetchSchedule? This would be
cleaner IMHO.
> Flag for generate to fetch only new pages to complement the -refetchonly flag
> -----------------------------------------------------------------------------
>
> Key: NUTCH-49
> URL: https://issues.apache.org/jira/browse/NUTCH-49
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher
> Reporter: Luke Baker
> Priority: Minor
> Attachments: fetchnewonly.patch
>
>
> It would be useful, especially for research/testing purposes, to have a flag
> for the FetchListTool that make sure to only include URLs in the fetchlist
> that have not already been fetched (according to the information from the
> webdb that you're generating the fetchlist from).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
