[
https://issues.apache.org/jira/browse/PIG-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887578#action_12887578
]
Alan Gates commented on PIG-1478:
---------------------------------
I don't understand the difference between launchStartedNotification() and
jobsSubmittedNotification().
When will outputCompletedNotification() be called? Only after the job is
completely done? What, if any, guarantees are we making on the order of this
relative to when PigRunner.run returns?
It isn't clear to me that launchCompleteNotification() is useful. Once the
launch has completed the user will start getting jobStartedNotification() calls.
> Add progress notification listener to PigRunner API
> ---------------------------------------------------
>
> Key: PIG-1478
> URL: https://issues.apache.org/jira/browse/PIG-1478
> Project: Pig
> Issue Type: Improvement
> Reporter: Richard Ding
> Assignee: Richard Ding
> Fix For: 0.8.0
>
> Attachments: PIG-1478.patch
>
>
> PIG-1333 added PigRunner API to allow Pig users and tools to get a
> status/stats object back after executing a Pig script. The new API, however,
> is synchronous (blocking). It's known that a Pig script can spawn tens (even
> hundreds) MR jobs and take hours to complete. Therefore it'll be nice to give
> progress feedback to the callers during the execution.
> The proposal is to add an optional parameter to the API:
> {code}
> public abstract class PigRunner {
> public static PigStats run(String[] args, PigProgressNotificationListener
> listener) {...}
> }
> {code}
> The new listener is defined as following:
> {code}
> package org.apache.pig.tools.pigstats;
> public interface PigProgressNotificationListener extends
> java.util.EventListener {
> // just before the launch of MR jobs for the script
> public void LaunchStartedNotification(int numJobsToLaunch);
> // number of jobs submitted in a batch
> public void jobsSubmittedNotification(int numJobsSubmitted);
> // a job is started
> public void jobStartedNotification(String assignedJobId);
> // a job is completed successfully
> public void jobFinishedNotification(JobStats jobStats);
> // a job is failed
> public void jobFailedNotification(JobStats jobStats);
> // a user output is completed successfully
> public void outputCompletedNotification(OutputStats outputStats);
> // updates the progress as percentage
> public void progressUpdatedNotification(int progress);
> // the script execution is done
> public void launchCompletedNotification(int numJobsSucceeded);
> }
> {code}
> Any thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
