Need a way to correlate MR jobs with Pig statements
---------------------------------------------------
Key: PIG-908
URL: https://issues.apache.org/jira/browse/PIG-908
Project: Pig
Issue Type: Wish
Reporter: Dmitriy V. Ryaboy
Complex Pig Scripts often generate many Map-Reduce jobs, especially with the
recent introduction of multi-store capabilities.
For example, the first script in the Pig tutorial produces 5 MR jobs.
There is currently very little support for debugging resulting jobs; if one of
the MR jobs fails, it is hard to figure out which part of the script it was
responsible for. Explain plans help, but even with the explain plan, a fair
amount of effort (and sometimes, experimentation) is required to correlate the
failing MR job with the corresponding PigLatin statements.
This ticket is created to discuss approaches to alleviating this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
