On Wed, Jan 22, 2014 at 03:39PM, Giridharan Kesavan wrote: > The work I mentioned earlier was related to Hive and Pig switching > to use Tez natively as compared to using Tez via the MR compat > layer built into Tez. I don't have an answer on when hive or pig would > get released with native Tez support, but I see hive getting tez support > in the next release. > Pig Tez integration work is still in progress and it might > get fixed in the next pig release. And there is also > a possibility this may not happen in the 0.8 timeline. > > Tez is an independent project which already supports mapreduce > via a compat. layer. AFAIK, it is not targeted just for pig or hive. > Furthermore, users of MR, hive and pig can already make use of > Tez by switching MR to run in tez-execution mode. > > To answer Cos's question on testing efforts, we could run simple > mapreduce like jobs (wordcount/ordered wordcount) to validate tez working.
Fair enough. Thanks for the clarification, Giri! > When hive and pig is ready we can enhance hive and pig integration test > suites > to make use of Tez api's and certify those application's. > > P.S: FWIW, the api is called doJobClientMagic() for the Tez-MR compat. > layer. :) Which proves my point, that I have no idea about it ;) > On Wed, Jan 22, 2014 at 2:34 PM, Konstantin Boudnik <[email protected]> wrote: > > > I was about asking the same question essentially. As of right now, there's > > no > > downstream components are using Tez. Hence, sure: we'll get it into the > > stack, > > but there's nothing to use it with not test it on. So, how we can make sure > > that the code doesn't bit-rot, etc.? > > > > And it isn't to give Giri any hard time, but just because there's no one > > else > > (as far as I know) in this community with any knowledge about that project. > > > > Cos > > > > P.S. I guess I wasn't totally accurate in my last statement. I know that > > Tez > > API has a method doClientMagic(). But that's all I know. Swear! > > > > On Wed, Jan 22, 2014 at 01:15PM, Andrew Purtell wrote: > > > My question is really asking for a guesstimate on when Tez, Hive, and Pig > > > all converge to a released state. I guess for Hive that would be release > > > 0.13. For Pig? Pardon me if I am mistaken but I thought Tez targets those > > > like Hive and Pig developers, not end users of MapReduce, Hive, or Pig. > > Can > > > we assume Hive and Pig when released with support for Tez will depend on > > > the same version of Tez? All of this is unlikely to happen within the > > > timeframe of Bigtop 0.8? Should we be talking about this for 0.9? > > > > > > > > > On Wed, Jan 22, 2014 at 11:10 AM, Giridharan Kesavan < > > > [email protected]> wrote: > > > > > > > Andrew, > > > > > > > > Hive on tez is fixed in the hive trunk and should be part of the next > > > > release, 0.13 > > > > Here is the link to the jira. > > > > https://issues.apache.org/jira/browse/HIVE-4660 > > > > > > > > Pig on tez integration is tracked here > > > > https://issues.apache.org/jira/browse/PIG-3446 > > > > > > > > > > > > -Giri > > > > > > > > > > > > On Wed, Jan 22, 2014 at 10:03 AM, Andrew Purtell <[email protected] > > > > >wrote: > > > > > > > > > At some point other components like Hive and Pig will want to pull in > > > > Tez, > > > > > but as far as I know no released versions use it yet. Can you shed > > some > > > > > insight on when that might be Giri? > > > > > > > > > > > > > > > On Tue, Jan 21, 2014 at 5:57 PM, Giridharan Kesavan < > > > > > [email protected]> wrote: > > > > > > > > > > > Can we include Tez-0.2.0 to the 0.8 BOM? > > > > > > > > > > > > I have working patch uploaded to BIGTOP-1179 > > > > > > > > > > > > -Giri > > > > > > > > > > > > > > > > > > On Tue, Jan 21, 2014 at 5:52 PM, Mark Grover <[email protected]> > > wrote: > > > > > > > > > > > > > Thanks for starting this email thread, Roman! I posted my > > opinion on > > > > > the > > > > > > > JIRA. > > > > > > > > > > > > > > > > > > > > > On Tue, Jan 21, 2014 at 10:27 AM, Andrew Purtell < > > > > [email protected] > > > > > > > >wrote: > > > > > > > > > > > > > > > Welcome back Roman. > > > > > > > > > > > > > > > > Likewise, I've been tied up with the day job but next month I > > can > > > > > > > > do BIGTOP-1105/BIGTOP-1148, and BIGTOP-1029 for HBase 0.98. > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jan 21, 2014 at 5:22 AM, Roman Shaposhnik < > > > > > > [email protected] > > > > > > > > >wrote: > > > > > > > > > > > > > > > > > Hi! > > > > > > > > > > > > > > > > > > First of all, I'd like to apologize for disappearing for > > > > > > > > > about a month from my RM duties of Bigtop 0.8.0 release. > > > > > > > > > New job brought so much excitement into my life > > > > > > > > > that I couldn't do justice to 0.8.0. > > > > > > > > > > > > > > > > > > The good news, though, is that I am almost back to > > > > > > > > > my regular schedule now and I really would like > > > > > > > > > to make sure that we start real work on Bigtop 0.8.0 > > > > > > > > > ASAP. > > > > > > > > > > > > > > > > > > To that end, I've updated: > > > > > > > > > https://issues.apache.org/jira/browse/BIGTOP-1110 > > > > > > > > > with a proposed BOM and platform details. > > > > > > > > > It doesn't have to be 100% accurate (we > > > > > > > > > can still tweak versions somewhat) but lets > > > > > > > > > make sure it aligns with the entire community. > > > > > > > > > > > > > > > > > > Please comment on the JIRA. Or on this very email > > > > > > > > > thread. Either one is good ;-) > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Roman. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Best regards, > > > > > > > > > > > > > > > > - Andy > > > > > > > > > > > > > > > > Problems worthy of attack prove their worth by hitting back. - > > Piet > > > > > > Hein > > > > > > > > (via Tom White) > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > CONFIDENTIALITY NOTICE > > > > > > NOTICE: This message is intended for the use of the individual or > > > > entity > > > > > to > > > > > > which it is addressed and may contain information that is > > confidential, > > > > > > privileged and exempt from disclosure under applicable law. If the > > > > reader > > > > > > of this message is not the intended recipient, you are hereby > > notified > > > > > that > > > > > > any printing, copying, dissemination, distribution, disclosure or > > > > > > forwarding of this communication is strictly prohibited. If you > > have > > > > > > received this communication in error, please contact the sender > > > > > immediately > > > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > > > > > > - Andy > > > > > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > > Hein > > > > > (via Tom White) > > > > > > > > > > > > > -- > > > > CONFIDENTIALITY NOTICE > > > > NOTICE: This message is intended for the use of the individual or > > entity to > > > > which it is addressed and may contain information that is confidential, > > > > privileged and exempt from disclosure under applicable law. If the > > reader > > > > of this message is not the intended recipient, you are hereby notified > > that > > > > any printing, copying, dissemination, distribution, disclosure or > > > > forwarding of this communication is strictly prohibited. If you have > > > > received this communication in error, please contact the sender > > immediately > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > > > > -- > > > Best regards, > > > > > > - Andy > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > > (via Tom White) > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
