[
https://issues.apache.org/jira/browse/NUTCH-2010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sebastian Nagel resolved NUTCH-2010.
------------------------------------
Fix Version/s: (was: 1.23)
Resolution: Won't Fix
NUTCH-3165 removed the Nutch web service including the FetchNode.
> Implement isFetchingInProgress Utility Function in Fetcher
> ----------------------------------------------------------
>
> Key: NUTCH-2010
> URL: https://issues.apache.org/jira/browse/NUTCH-2010
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.10
> Reporter: Lewis John McGibbney
> Priority: Major
>
> The aim here is to stop (without killing) a Nutch crawl if the data being
> fetched is not of value to the user. The user can infer this by implementing
> some visualization on top of the backported REST API for Nutch trunk (could
> probably also do with with 2.X REST API as well tbh).
> I suggest that we implement a convenience utility function in potentially
> Fetcher.java which would looking something like the following
> {code}
> public static boolean isFetchingInProgress() {
> return fetchingInProgress;
> }
> {code}
> The fetchingInProgress should be set to tru whenever fetcher threads are
> working and should be set to false whenever all fetcher threads are
> unoccupied and back in the pool vacant.
> This would be a powerful mechanism for determining if a crawl could be
> stopped without corrupting data as currently happens when a fetcher task is
> interrupted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
