Gunther Hagleitner updated PIG-781:

    Attachment: partial_failure.patch

The new patch does same as above (report on the failed and succeeded jobs), but 

   * Returns a list of exec jobs, one for each store, so that embedded programs 
can iterate through results and determine success and failures
   * Adds a flag "-F" or "-stop_on_failure" that causes an exception on the 
first failure which will cause the processing to stop.
   * Returns 2 when all jobs fail or when the stop_on_failure flag is 
specified. Returns 3 if some jobs passed and others failed.

> Error reporting for failed MR jobs
> ----------------------------------
>                 Key: PIG-781
>                 URL: https://issues.apache.org/jira/browse/PIG-781
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Gunther Hagleitner
>         Attachments: partial_failure.patch, partial_failure.patch
> If we have multiple MR jobs to run and some of them fail the behavior of the 
> system is to not stop on the first failure but to keep going. That way jobs 
> that do not depend on the failed job might still succeed.
> The question is to how best report this scenario to a user. How do we tell 
> which jobs failed and which didn't?
> One way could be to tie jobs to stores and report which store locations won't 
> have data and which ones do.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to