Thanks Cheolsoo! My opinion is provide backward compatibility for PigStats is a must, otherwise it could be a havoc. I imagine PigStats is widely used by Pig users via PigRunner and PPNL interface. People use PigStats to collect MR job details of the Pig job. Though PigStats is marked for Evolving, this is mostly for extending PigStats, not limiting PigStats as PIG-3419 did. Even if we really need to change, we need to very well communicate with users over time, Pig 0.12 is not an option.
PIG-3457 is trying to provide a backward compatibility way for PigStats, but just like Cheolsoo said, it is far from ideal. I now tend to agree Rohini's suggestion on PIG-3419, rollback PIG-3419, until we find a better way. Seems PIG-3419 is a little premature. Besides the above mentioned PigStats issue, I've already found 2 additional issues: 1. "explain" shows unoptimized logical plan instead of optimized one 2. HangingJobKiller is removed How does others think? Thanks, Daniel On Mon, Sep 30, 2013 at 9:51 AM, Cheolsoo Park <[email protected]> wrote: > Hi devs, > > PIG-3419 <https://issues.apache.org/jira/browse/PIG-3419> broke backward > compatibility for downstream applications such as Oozie, and > PIG-3457<https://issues.apache.org/jira/browse/PIG-3457> is > trying to fix it. > > In summary, we need to keep the old MR-specific JobStats and PigStats for > backward compatibility sadly because they're currently exposed in several > user-facing API including PigRunner, PigServer, etc. However, this defeats > the purpose of PIG-3419 > <https://issues.apache.org/jira/browse/PIG-3419> because > we cannot implement non-MR execution engines in the back-end if the > front-end API is tied to MR. > > Daniel and I were having a discussion in the jira, but it seems more > complicated than I thought. I am wondering whether anyone has a good > suggestion on how to solve it. This is blocking the 0.12 release. > > Thanks, > Cheolsoo > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
