[
https://issues.apache.org/jira/browse/NUTCH-866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Nioche resolved NUTCH-866.
---------------------------------
Resolution: Not A Problem
> STOP Nutch without breaking the crawled data
> --------------------------------------------
>
> Key: NUTCH-866
> URL: https://issues.apache.org/jira/browse/NUTCH-866
> Project: Nutch
> Issue Type: New Feature
> Reporter: Pham Tuan Minh
> Fix For: 2.0
>
>
> How we can stop running nutch instance in local mode and in reducer mode
> without breaking the crawled data?
> For example, you push a list of site that take a long time to complete crawl;
> then you want to stop nutch instance suddenly ...
> - For local mode, I suggest as below
> We create a stop.txt file in specific directory, then for a piece of time,
> nutch instance will check whether this file existed or not; if existed, nutch
> instance will stop itself normally
> - For reducer mode, may we use zookeper to keep state of each instance?
> Any other suggestion?
> Thanks,
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
