The work I mentioned earlier was related to Hive and Pig switching to use Tez natively as compared to using Tez via the MR compat layer built into Tez. I don't have an answer on when hive or pig would get released with native Tez support, but I see hive getting tez support in the next release. Pig Tez integration work is still in progress and it might get fixed in the next pig release. And there is also a possibility this may not happen in the 0.8 timeline.
Tez is an independent project which already supports mapreduce via a compat. layer. AFAIK, it is not targeted just for pig or hive. Furthermore, users of MR, hive and pig can already make use of Tez by switching MR to run in tez-execution mode. To answer Cos's question on testing efforts, we could run simple mapreduce like jobs (wordcount/ordered wordcount) to validate tez working. When hive and pig is ready we can enhance hive and pig integration test suites to make use of Tez api's and certify those application's. P.S: FWIW, the api is called doJobClientMagic() for the Tez-MR compat. layer. :) -Giri On Wed, Jan 22, 2014 at 2:34 PM, Konstantin Boudnik <[email protected]> wrote: > I was about asking the same question essentially. As of right now, there's > no > downstream components are using Tez. Hence, sure: we'll get it into the > stack, > but there's nothing to use it with not test it on. So, how we can make sure > that the code doesn't bit-rot, etc.? > > And it isn't to give Giri any hard time, but just because there's no one > else > (as far as I know) in this community with any knowledge about that project. > > Cos > > P.S. I guess I wasn't totally accurate in my last statement. I know that > Tez > API has a method doClientMagic(). But that's all I know. Swear! > > On Wed, Jan 22, 2014 at 01:15PM, Andrew Purtell wrote: > > My question is really asking for a guesstimate on when Tez, Hive, and Pig > > all converge to a released state. I guess for Hive that would be release > > 0.13. For Pig? Pardon me if I am mistaken but I thought Tez targets those > > like Hive and Pig developers, not end users of MapReduce, Hive, or Pig. > Can > > we assume Hive and Pig when released with support for Tez will depend on > > the same version of Tez? All of this is unlikely to happen within the > > timeframe of Bigtop 0.8? Should we be talking about this for 0.9? > > > > > > On Wed, Jan 22, 2014 at 11:10 AM, Giridharan Kesavan < > > [email protected]> wrote: > > > > > Andrew, > > > > > > Hive on tez is fixed in the hive trunk and should be part of the next > > > release, 0.13 > > > Here is the link to the jira. > > > https://issues.apache.org/jira/browse/HIVE-4660 > > > > > > Pig on tez integration is tracked here > > > https://issues.apache.org/jira/browse/PIG-3446 > > > > > > > > > -Giri > > > > > > > > > On Wed, Jan 22, 2014 at 10:03 AM, Andrew Purtell <[email protected] > > > >wrote: > > > > > > > At some point other components like Hive and Pig will want to pull in > > > Tez, > > > > but as far as I know no released versions use it yet. Can you shed > some > > > > insight on when that might be Giri? > > > > > > > > > > > > On Tue, Jan 21, 2014 at 5:57 PM, Giridharan Kesavan < > > > > [email protected]> wrote: > > > > > > > > > Can we include Tez-0.2.0 to the 0.8 BOM? > > > > > > > > > > I have working patch uploaded to BIGTOP-1179 > > > > > > > > > > -Giri > > > > > > > > > > > > > > > On Tue, Jan 21, 2014 at 5:52 PM, Mark Grover <[email protected]> > wrote: > > > > > > > > > > > Thanks for starting this email thread, Roman! I posted my > opinion on > > > > the > > > > > > JIRA. > > > > > > > > > > > > > > > > > > On Tue, Jan 21, 2014 at 10:27 AM, Andrew Purtell < > > > [email protected] > > > > > > >wrote: > > > > > > > > > > > > > Welcome back Roman. > > > > > > > > > > > > > > Likewise, I've been tied up with the day job but next month I > can > > > > > > > do BIGTOP-1105/BIGTOP-1148, and BIGTOP-1029 for HBase 0.98. > > > > > > > > > > > > > > > > > > > > > On Tue, Jan 21, 2014 at 5:22 AM, Roman Shaposhnik < > > > > > [email protected] > > > > > > > >wrote: > > > > > > > > > > > > > > > Hi! > > > > > > > > > > > > > > > > First of all, I'd like to apologize for disappearing for > > > > > > > > about a month from my RM duties of Bigtop 0.8.0 release. > > > > > > > > New job brought so much excitement into my life > > > > > > > > that I couldn't do justice to 0.8.0. > > > > > > > > > > > > > > > > The good news, though, is that I am almost back to > > > > > > > > my regular schedule now and I really would like > > > > > > > > to make sure that we start real work on Bigtop 0.8.0 > > > > > > > > ASAP. > > > > > > > > > > > > > > > > To that end, I've updated: > > > > > > > > https://issues.apache.org/jira/browse/BIGTOP-1110 > > > > > > > > with a proposed BOM and platform details. > > > > > > > > It doesn't have to be 100% accurate (we > > > > > > > > can still tweak versions somewhat) but lets > > > > > > > > make sure it aligns with the entire community. > > > > > > > > > > > > > > > > Please comment on the JIRA. Or on this very email > > > > > > > > thread. Either one is good ;-) > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Roman. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best regards, > > > > > > > > > > > > > > - Andy > > > > > > > > > > > > > > Problems worthy of attack prove their worth by hitting back. - > Piet > > > > > Hein > > > > > > > (via Tom White) > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > CONFIDENTIALITY NOTICE > > > > > NOTICE: This message is intended for the use of the individual or > > > entity > > > > to > > > > > which it is addressed and may contain information that is > confidential, > > > > > privileged and exempt from disclosure under applicable law. If the > > > reader > > > > > of this message is not the intended recipient, you are hereby > notified > > > > that > > > > > any printing, copying, dissemination, distribution, disclosure or > > > > > forwarding of this communication is strictly prohibited. If you > have > > > > > received this communication in error, please contact the sender > > > > immediately > > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > > > > > - Andy > > > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein > > > > (via Tom White) > > > > > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or > entity to > > > which it is addressed and may contain information that is confidential, > > > privileged and exempt from disclosure under applicable law. If the > reader > > > of this message is not the intended recipient, you are hereby notified > that > > > any printing, copying, dissemination, distribution, disclosure or > > > forwarding of this communication is strictly prohibited. If you have > > > received this communication in error, please contact the sender > immediately > > > and delete it from your system. Thank You. > > > > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
