Yes i did change PIG/ivy/libraries.propeties to compile it with tez-0.7.0 and also changed pig to compile with Hadoop-core-2.6.0.
I could not use tez-0.5.2 because it was not compatible with Hadoop-2.6.0. I'm compiling my code of PIG using same command: ant clean jar -Dhadoopversion=23 On Wed, Sep 2, 2015 at 2:36 PM, Jianfeng (Jeff) Zhang < [email protected]> wrote: > > Not sure how did you compile pig with tez 0.7.0, did you change the tez > version in PIG/ivy/libraries.propeties ? > > And make sure you build pig with hadoop version, by default, pig build > with hadoop-1.x. Use the following command to build pig with hadoop-2.x > > >> ant clean jar -Dhadoopversion=23 > > > > Best Regard, > Jeff Zhang > > > From: Sandeep Kumar <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Wednesday, September 2, 2015 at 4:27 PM > To: "[email protected]" <[email protected]> > Subject: Re: Pig(0.14.0) on Tez(0.7.0) > > Hi Jeff, > > The cloudera Hadoop is using guava-11.0.2.jar. > I've also exported one environment variable before running pig: > > export HADOOP_USER_CLASSPATH_FIRST=true > > Should i recompile my PIG code with new guava jar? > > Regards, > Sandeep > > On Wed, Sep 2, 2015 at 1:20 PM, Jianfeng (Jeff) Zhang < > [email protected]> wrote: > >> >> It looks like classpath issue due guava version. Could you check the >> guava version CDH 5.4.4 is using ? AFAIK, Tez use guava 11.02 and pig use >> guava 11.0 >> >> >> >> Best Regard, >> Jeff Zhang >> >> >> From: Sandeep Kumar <[email protected]> >> Reply-To: "[email protected]" <[email protected]> >> Date: Wednesday, September 2, 2015 at 2:28 PM >> To: "[email protected]" <[email protected]> >> Subject: Pig(0.14.0) on Tez(0.7.0) >> >> Hi Guys, >> >> I've a pig script which is as follows: >> >> map = LOAD 'input/' using >> com.RawPigLoader('conf/Map.xml','conf/R360MapSignalling.json','csv'); >> normalized_map_data = foreach r360map generate flatten(com.Map($0..)); >> >> data_grouped_over_event_time = GROUP normalized_map_data by >> (((startTime/1000) / 3600) * 3600) PARTITION BY com.CustomTimePartitioner; >> final_data = foreach data_grouped_over_event_time generate flatten($1); >> >> store final_data into 'tmp/200MB' using com.StorageModel(); >> >> >> I'm trying to run my Pig job using Tez but i'm facing an exception which >> is attached. I searched over net and found some vague solutions. I've only >> two configuration entries in my tez-site.xml which are as follows: >> >> <property> >> <name>tez.lib.uris</name> >> <value>hdfs://node3:8020/user/hdfs/tez-0.7.0.tar.gz</value> >> </property> >> <property> >> <description>URL for where the Tez UI is hosted</description> >> <name>tez.tez-ui.history-url.base</name> >> <value>http://node4:9999/tez-ui/</value> >> </property> >> >> Is there anything else which i'm missing and supposed to be added in Tez >> configuration file. >> I've compiled PIG-0.14.0 manually with Tez-0.7.0 whereas it provided with >> Tez-0.5.2 but the exception is coming in both the versions of tez. >> >> Following are the version of softwares i'm using. >> >> HadoopVersion: >> 2.6.0-cdh5.4.4 >> >> PigVersion: >> 0.14.0-SNAPSHOT >> >> TezVersion: >> 0.7.0 >> >> >> >> >> Regards, >> Sandeep >> > >
