Hi Aniket, Thanks for raising a concern. I agree that we should not break api compatibility between 0.13 and onwards.
Currently, I don't think we've introduced any incompatibility between trunk and tez branch. As for Ambrose and Lipstick, we haven't decided what to do with PPNL yet. But I will start working on it soon since I cannot deploy Pig on Tez without Lipstick at Netflix. I will keep you in the loop so that you can ensure whatever changes that I will make will be also fine with Ambrose. Does this sound good to you? Thanks, Cheolsoo On Fri, Feb 14, 2014 at 4:27 PM, Aniket Mokashi <aniket...@gmail.com> wrote: > Typos (hit send too soon): > I meant, comment on api compatibility of pig-tez and pig-trunk (including > Utils etc.) > > I would like to make sure 0.13 is compatible with the next version to avoid > ramping up tools that depend on pig (Ambrose etc). > > ~Aniket > > > On Fri, Feb 14, 2014 at 1:23 PM, Aniket Mokashi <aniket...@gmail.com> > wrote: > > > Another thing: Pig-trunk has loads of incompatible interface changes > > already. It would be great if someone from pig-tez dev (Daniel et al) can > > comment on compatibility of these with pig-trunk? If they are not > > compatible or if there is any api refactor on wish list, can we please > open > > jiras for it? We should try to ensure that there is NO api > incompatibility > > between 0.13 and 0.14 (when it is out future). > > > > Thanks, > > Aniket > > > > > > On Thu, Feb 13, 2014 at 8:09 PM, Koji Noguchi <knogu...@yahoo-inc.com > >wrote: > > > >> For 0.13, can we get > >> > >> PIG-3508 (covered by Daniel's PIG-3545) > >> Otherwise, "explain" would show pre-optimized logical plan in 0.13. > >> > >> > I can wrap a 0.12.1 release in a couple of weeks. > >> > > >> Great! > >> > >> Koji > >> > >> > >> > >> On Feb 13, 2014, at 7:11 PM, Daniel Dai <da...@hortonworks.com> wrote: > >> > >> > Ok, it seems to leave no choice for a sooner 0.14.0 release then. > >> > > >> > On Thu, Feb 13, 2014 at 3:50 PM, Julien Le Dem <jul...@ledem.net> > >> wrote: > >> >> We can't use SNAPSHOT releases as dependencies of our official > >> releases. > >> >> They can change and disappear. > >> >> Let's use the official release of hive when it's available. > >> >> > >> >> On Feb 13, 2014, at 3:33 PM, Daniel Dai wrote: > >> >> > >> >>> I can also use a hive 0.13.0 SNAPSHOT in PIG-3558, and then switch > to > >> >>> a released version later. Seems that's much easier. > >> >>> > >> >>> On Thu, Feb 13, 2014 at 2:35 PM, Daniel Dai <da...@hortonworks.com> > >> wrote: > >> >>>> That works. I will roll a 0.14.0 release as soon as Hive 0.13.0 > >> released. > >> >>>> > >> >>>> On Thu, Feb 13, 2014 at 2:25 PM, Julien Le Dem <jul...@ledem.net> > >> wrote: > >> >>>>> I think it would be OK to release Pig 0.14 soon after if you want > >> new features in. > >> >>>>> It's not more work to release 0.14 than 0.13.1 and that keeps the > >> semantic versioning. > >> >>>>> > >> >>>>> On Feb 13, 2014, at 1:29 PM, Daniel Dai wrote: > >> >>>>> > >> >>>>>> Hive-0.13.0 will be release sometime March AFAIK. If we cannot > make > >> >>>>>> PIG-3558 to Pig 0.13.0, I still want to put in Pig 0.13.1, though > >> it > >> >>>>>> is a new feature. > >> >>>>>> > >> >>>>>> Other than those listed Jiras, there are also several unit > >> >>>>>> tests/system tests failures (eg, TestHBaseStorage) we need to > fix. > >> >>>>>> > >> >>>>>> On Thu, Feb 13, 2014 at 11:45 AM, Aniket Mokashi < > >> aniket...@gmail.com> wrote: > >> >>>>>>> Sounds good. Do we know a timeline for Hive-0.13 release? If its > >> not too > >> >>>>>>> far, we can try to do Pig-0.13 it a few weeks after that. > >> >>>>>>> > >> >>>>>>> So far we have following jiras to wrap up before Pig-0.13 > release. > >> >>>>>>> PIG-3591 > >> >>>>>>> PIG-3740 > >> >>>>>>> PIG-3745 > >> >>>>>>> PIG-3347 > >> >>>>>>> PIG-3731 > >> >>>>>>> PIG-3558 > >> >>>>>>> > >> >>>>>>> Any other? We can start marking jiras with corresponding version > >> numbers. > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> On Thu, Feb 13, 2014 at 9:58 AM, Daniel Dai < > >> da...@hortonworks.com> wrote: > >> >>>>>>> > >> >>>>>>>> I'd like to get PIG-3558 in, which depends on HIVE-0.13.0 > >> release in > >> >>>>>>>> weeks. Otherwise I am fine with a Pig 0.13.0 release. > >> >>>>>>>> > >> >>>>>>>> I can wrap a 0.12.1 release in a couple of weeks. > >> >>>>>>>> > >> >>>>>>>> Thanks, > >> >>>>>>>> Daniel > >> >>>>>>>> > >> >>>>>>>> On Thu, Feb 13, 2014 at 3:38 AM, Gianmarco De Francisci Morales > >> >>>>>>>> <g...@apache.org> wrote: > >> >>>>>>>>> +1 on releasing a 0.13 and if somebody feels strongly about > >> releasing a > >> >>>>>>>>> 0.12.1 that fixes PIG-3492 I am +1 on that too. > >> >>>>>>>>> > >> >>>>>>>>> -- > >> >>>>>>>>> Gianmarco > >> >>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>>> On 13 February 2014 03:08, Dmitriy Ryaboy <dvrya...@gmail.com > > > >> wrote: > >> >>>>>>>>> > >> >>>>>>>>>> So I think we agree that we should branch 0.13 at this point, > >> right? > >> >>>>>>>>>> > >> >>>>>>>>>> (and possibly look at releasing an incremental bump to 12 or > >> 10? I'm not > >> >>>>>>>>>> sure what should be included there but I support the general > >> idea). > >> >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> On Thu, Feb 6, 2014 at 2:22 PM, Koji Noguchi < > >> knogu...@yahoo-inc.com> > >> >>>>>>>>>> wrote: > >> >>>>>>>>>> > >> >>>>>>>>>>>> Releasing 0.13 and 0.10.1 is totally independent in my > >> opinion > >> >>>>>>>>>>>> > >> >>>>>>>>>>> I should have referenced my previous request on including > >> 0.10.1 on > >> >>>>>>>> the > >> >>>>>>>>>>> top release page. > >> >>>>>>>>>>> > http://www.mail-archive.com/dev@pig.apache.org/msg20629.html > >> >>>>>>>>>>> > >> >>>>>>>>>>>> By minor I meant 0.13 > >> >>>>>>>>>>>> 0.10.1 is a bug fix release. > >> >>>>>>>>>>>> as in Major.Minor.BugFix > >> >>>>>>>>>>>> > >> >>>>>>>>>>> I see. Then I should have said, > >> >>>>>>>>>>> "but I'd like to request we make BugFix releases more > often." > >> >>>>>>>>>>> > >> >>>>>>>>>>> Thanks for correcting my mistake. > >> >>>>>>>>>>> > >> >>>>>>>>>>> Koji > >> >>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>>> On Feb 6, 2014, at 5:05 PM, Julien Le Dem <jul...@ledem.net > > > >> wrote: > >> >>>>>>>>>>> > >> >>>>>>>>>>>> Releasing 0.13 and 0.10.1 is totally independent in my > >> opinion. > >> >>>>>>>>>>>> It just takes the time of a committer that needs the > release > >> to > >> >>>>>>>> happen > >> >>>>>>>>>>> to do it. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> By minor I meant 0.13 > >> >>>>>>>>>>>> 0.10.1 is a bug fix release. > >> >>>>>>>>>>>> as in Major.Minor.BugFix > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> Our Major version is still 0 > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> On Feb 6, 2014, at 1:43 PM, Koji Noguchi wrote: > >> >>>>>>>>>>>> > >> >>>>>>>>>>>>>> To add to the discussion, I think we should release more > >> often, > >> >>>>>>>> based > >> >>>>>>>>>>> on time elapsed rather than volume of change. > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>> I don't have preference on the frequency, but I'd like to > >> request > >> >>>>>>>> we > >> >>>>>>>>>>> make minor releases more often. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> At this moment, stable pig release (to me) is still > 0.10.1. > >> >>>>>>>>>>>>> 0.11.1 and 0.12.0 both have regression bug PIG-3492 that > >> caused > >> >>>>>>>>>>> multiple production pig scripts in our clusters to fail > >> randomly. > >> >>>>>>>>>>>>> (unless user is disabling ColumnMapKeyPrune) > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> If releasing 0.13 means 0.10.1 gets kicked out from the > >> front > >> >>>>>>>> release > >> >>>>>>>>>>> list, I'd like to see minor release on 0.11 or 0.12 first. > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> Koji > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>> On Feb 6, 2014, at 4:25 PM, Cheolsoo Park < > >> piaozhe...@gmail.com> > >> >>>>>>>>>>>>> wrote: > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>>>> +1 to 0.13 release. Why not if someone is volunteering? > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>> On Thu, Feb 6, 2014 at 4:06 PM, Julien Le Dem < > >> jul...@ledem.net> > >> >>>>>>>>>>> wrote: > >> >>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> To add to the discussion, I think we should release more > >> often, > >> >>>>>>>>>> based > >> >>>>>>>>>>> on > >> >>>>>>>>>>>>>>> time elapsed rather than volume of change. > >> >>>>>>>>>>>>>>> The more often we release, the easier it is to release. > >> >>>>>>>>>>>>>>> Also that makes it easier for contributors to use their > >> own > >> >>>>>>>>>>> contributions > >> >>>>>>>>>>>>>>> in official releases. > >> >>>>>>>>>>>>>>> It is also probably a good idea to have a clean starting > >> point > >> >>>>>>>>>> before > >> >>>>>>>>>>>>>>> merging the Tez branch > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> That said, I think those changes by themselves are > enough > >> to > >> >>>>>>>>>> warrant a > >> >>>>>>>>>>>>>>> minor release. > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> Julien > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> On Feb 6, 2014, at 12:24 PM, Dmitriy Ryaboy wrote: > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> Major updates since we release 12 that are currently in > >> trunk: > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> - lazy output (don't generate empty part files) > >> >>>>>>>>>>>>>>>> - jar caching optimization > >> >>>>>>>>>>>>>>>> - automatic local mode for small job (big wall-clock > >> wins for > >> >>>>>>>>>>> long-tail > >> >>>>>>>>>>>>>>>> jobs) > >> >>>>>>>>>>>>>>>> - improved support for BigInteger, BigDecimal > >> >>>>>>>>>>>>>>>> - hbase loader improvements > >> >>>>>>>>>>>>>>>> - debug mode that leaves temp files around for > >> examination (!) > >> >>>>>>>>>>>>>>>> - fixes to a few nasty bugs (PIG-3641) > >> >>>>>>>>>>>>>>>> - pluggable execution engine allowing work like Tez and > >> Spork > >> >>>>>>>>>>>>>>>> - .. and more > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> I'd say this justifies a release. > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> D > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>> On Wed, Feb 5, 2014 at 3:55 PM, Aniket Mokashi < > >> >>>>>>>>>> aniket...@gmail.com> > >> >>>>>>>>>>>>>>> wrote: > >> >>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> List I mentioned is pending tasks before we can make a > >> release. > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> A complete list of contributions can be seen at - > >> >>>>>>>>>>>>>>>>> > >> http://svn.apache.org/viewvc/pig/trunk/CHANGES.txt?view=markup > >> >>>>>>>> . > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> Some of the things that make it a good candidate for a > >> release- > >> >>>>>>>>>>>>>>>>> - PIG-3419 (has several backwards incompatible api > >> changes) > >> >>>>>>>>>>>>>>>>> - PIG-2672 > >> >>>>>>>>>>>>>>>>> - PIG-3642 > >> >>>>>>>>>>>>>>>>> - PIG-3463 > >> >>>>>>>>>>>>>>>>> - PIG-3511 > >> >>>>>>>>>>>>>>>>> - PIG-3657 > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> Thanks, > >> >>>>>>>>>>>>>>>>> Aniket > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> On Wed, Feb 5, 2014 at 3:23 PM, Olga Natkovich < > >> >>>>>>>>>>> onatkov...@yahoo.com> > >> >>>>>>>>>>>>>>>>> wrote: > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> Just going by the list that Aniket provided, I don't > >> really > >> >>>>>>>> see > >> >>>>>>>>>>> enough > >> >>>>>>>>>>>>>>>>> for > >> >>>>>>>>>>>>>>>>>> a full release. Two mentioned JIRAs are doc updates > >> and one > >> >>>>>>>> is a > >> >>>>>>>>>>> bug > >> >>>>>>>>>>>>>>> fix > >> >>>>>>>>>>>>>>>>>> that was ported into Pig 12. > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> On Wednesday, February 5, 2014 3:13 PM, Aniket > Mokashi > >> < > >> >>>>>>>>>>>>>>>>>> aniket...@gmail.com> wrote: > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> Hi All, > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> A good number of improvements and bug fixes have gone > >> into > >> >>>>>>>> trunk > >> >>>>>>>>>>>>>>>>> recently. > >> >>>>>>>>>>>>>>>>>> I'd like to know if we can roll out a Pig 0.13 > release > >> around > >> >>>>>>>>>>>>>>> mid-March? > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> I am aware that we are planning to merge tez branch > >> into trunk > >> >>>>>>>>>>> soon. > >> >>>>>>>>>>>>>>>>>> However, making a release before tez branch is merged > >> will be > >> >>>>>>>>>>> good. Any > >> >>>>>>>>>>>>>>>>>> objections? > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> Following are few jiras we need to wrap up before > 0.13 > >> >>>>>>>> release- > >> >>>>>>>>>>>>>>>>>> PIG-3591 > >> >>>>>>>>>>>>>>>>>> PIG-3740 > >> >>>>>>>>>>>>>>>>>> PIG-3745 > >> >>>>>>>>>>>>>>>>>> PIG-3347 > >> >>>>>>>>>>>>>>>>>> PIG-3731 > >> >>>>>>>>>>>>>>>>>> Any other? > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>>> Thanks, > >> >>>>>>>>>>>>>>>>>> Aniket > >> >>>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>>>> -- > >> >>>>>>>>>>>>>>>>> "...:::Aniket:::... Quetzalco@tl" > >> >>>>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>>>> > >> >>>>>>>>>>>>> > >> >>>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>> > >> >>>>>>>> -- > >> >>>>>>>> CONFIDENTIALITY NOTICE > >> >>>>>>>> NOTICE: This message is intended for the use of the individual > >> or entity to > >> >>>>>>>> which it is addressed and may contain information that is > >> confidential, > >> >>>>>>>> privileged and exempt from disclosure under applicable law. If > >> the reader > >> >>>>>>>> of this message is not the intended recipient, you are hereby > >> notified that > >> >>>>>>>> any printing, copying, dissemination, distribution, disclosure > or > >> >>>>>>>> forwarding of this communication is strictly prohibited. If you > >> have > >> >>>>>>>> received this communication in error, please contact the sender > >> immediately > >> >>>>>>>> and delete it from your system. Thank You. > >> >>>>>>>> > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> -- > >> >>>>>>> "...:::Aniket:::... Quetzalco@tl" > >> >>>>>> > >> >>>>>> -- > >> >>>>>> CONFIDENTIALITY NOTICE > >> >>>>>> NOTICE: This message is intended for the use of the individual or > >> entity to > >> >>>>>> which it is addressed and may contain information that is > >> confidential, > >> >>>>>> privileged and exempt from disclosure under applicable law. If > the > >> reader > >> >>>>>> of this message is not the intended recipient, you are hereby > >> notified that > >> >>>>>> any printing, copying, dissemination, distribution, disclosure or > >> >>>>>> forwarding of this communication is strictly prohibited. If you > >> have > >> >>>>>> received this communication in error, please contact the sender > >> immediately > >> >>>>>> and delete it from your system. Thank You. > >> >>>>> > >> >>> > >> >>> -- > >> >>> CONFIDENTIALITY NOTICE > >> >>> NOTICE: This message is intended for the use of the individual or > >> entity to > >> >>> which it is addressed and may contain information that is > >> confidential, > >> >>> privileged and exempt from disclosure under applicable law. If the > >> reader > >> >>> of this message is not the intended recipient, you are hereby > >> notified that > >> >>> any printing, copying, dissemination, distribution, disclosure or > >> >>> forwarding of this communication is strictly prohibited. If you have > >> >>> received this communication in error, please contact the sender > >> immediately > >> >>> and delete it from your system. Thank You. > >> >> > >> > > >> > -- > >> > CONFIDENTIALITY NOTICE > >> > NOTICE: This message is intended for the use of the individual or > >> entity to > >> > which it is addressed and may contain information that is > confidential, > >> > privileged and exempt from disclosure under applicable law. If the > >> reader > >> > of this message is not the intended recipient, you are hereby notified > >> that > >> > any printing, copying, dissemination, distribution, disclosure or > >> > forwarding of this communication is strictly prohibited. If you have > >> > received this communication in error, please contact the sender > >> immediately > >> > and delete it from your system. Thank You. > >> > >> > > > > > > -- > > "...:::Aniket:::... Quetzalco@tl" > > > > > > -- > "...:::Aniket:::... Quetzalco@tl" >
