Hi Ben, There is no way to stop a job unless you can afford loosing all data being processed (or about to be processed) within that stage of the crawl cycle. Lewis
On Wed, Jun 14, 2017 at 7:42 AM, <[email protected]> wrote: > > From: Ben Vachon <[email protected]> > To: [email protected] > Cc: > Bcc: > Date: Mon, 12 Jun 2017 10:19:24 -0400 > Subject: Stop Local Job Threads > Hi all, > > I'm running Nutch 2.3.1 over a standalone HBase instance with no yarn or > hdfs. This means that the jobs get run through > org.apache.hadoop.mapred.LocalJobRunner which doesn't support killing > mapred tasks. I've set it up so that all of the nutch threads get run in > the same ThreadGroup and I've been trying to iterate over all the threads > in that group and call interrupt on them, but they seem to just be > swallowing the exceptions. I've been testing stopping during the fetch job > specifically. > > Does any one know how to stop a locally run Nutch mapreduce job before it > completes? > >
