To be clear, the unless mapreduce.framework.name is set to yarn-tez, MR jobs will continue to run on MR.
However, the client side MR code may see issues if the Hadoop version for MR and Tez are different and there is an incompatible change in the Hadoop between those versions. E.g. there was an incompatible change made in the deprecation code in Hadoop between 2.2 and 2.4. So if the MR code depends on 2.2 hadoop libs and tez (latest) uses 2.4 then the presence of those libs in the client classpath will hit that incompatible change. This is a client side only issue. Bikas *From:* Bala Krishna Gangisetty [mailto:[email protected]] *Sent:* Monday, July 07, 2014 5:51 PM *To:* [email protected] *Subject:* Re: MRv2 and Tez co-existence Thanks Hitesh for your inputs. I've not come across any issues yet. So, I can safely assume that putting Tez jars in Hadoop class path will not cause the map reduce programs to use Tez framework unless it is enabled. Let me know if my understanding it not correct. --Bala G. On Mon, Jul 7, 2014 at 4:10 PM, Hitesh Shah <[email protected]> wrote: Hi For the most part, there should be no issues as most dependencies that Tez pulls in are compatible with the hadoop version that it is compiled with ( 2.2 or higher ). The major issue to be aware of is that you should compile Tez against the same version of hadoop/mapreduce that is deployed on your cluster. The tez dependency jars contain both 3rd party deps as well as hadoop jars ( hdfs, common, yarn client-side and mapreduce client-side ) - if there is a version mismatch, this may cause a problem when the tez directory is added to the hadoop classpath. Have you seen any issues? If yes, could you provide more details? thanks — Hitesh On Jul 7, 2014, at 3:44 PM, Bala Krishna Gangisetty <[email protected]> wrote: > I'm wondering, from operational point of view, are there any specifics that need special attention to make MRv2 and Tez frameworks coexist in harmony? I heard that putting Tez jars in Hadoop class path would impact the mapred behavior, even when Tez is not enabled (either through mapred-site.xml, or Hive). Could someone throw more light and share thoughts on it? > > --Bala G. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
