Sebastian Nagel created NUTCH-2518:

             Summary: Must check return value of job.waitForCompletion()
                 Key: NUTCH-2518
             Project: Nutch
          Issue Type: Bug
          Components: crawldb, fetcher, generator, hostdb, linkdb
    Affects Versions: 1.15
            Reporter: Sebastian Nagel
             Fix For: 1.15

The return value of job.waitForCompletion() of the new MapReduce API 
(NUTCH-2375) must always be checked. If it's not true, the job has been failed 
or killed. Accordingly, the program
- should not proceed with further jobs/steps
- must clean-up temporary data, unlock CrawlDB, etc.
- exit with non-zero exit value, so that scripts running the crawl workflow can 
handle the failure

Cf. NUTCH-2076, NUTCH-2442, [NUTCH-2375 PR 

This message was sent by Atlassian JIRA

Reply via email to