>>> I could not use tez-0.5.2 because it was not compatible with Hadoop-2.6.0.
What incompatible do you see ? Best Regard, Jeff Zhang From: Sandeep Kumar <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, September 2, 2015 at 5:18 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Pig(0.14.0) on Tez(0.7.0) Yes i did change PIG/ivy/libraries.propeties to compile it with tez-0.7.0 and also changed pig to compile with Hadoop-core-2.6.0. I could not use tez-0.5.2 because it was not compatible with Hadoop-2.6.0. I'm compiling my code of PIG using same command: ant clean jar -Dhadoopversion=23 On Wed, Sep 2, 2015 at 2:36 PM, Jianfeng (Jeff) Zhang <[email protected]<mailto:[email protected]>> wrote: Not sure how did you compile pig with tez 0.7.0, did you change the tez version in PIG/ivy/libraries.propeties ? And make sure you build pig with hadoop version, by default, pig build with hadoop-1.x. Use the following command to build pig with hadoop-2.x >> ant clean jar -Dhadoopversion=23 Best Regard, Jeff Zhang From: Sandeep Kumar <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, September 2, 2015 at 4:27 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: Pig(0.14.0) on Tez(0.7.0) Hi Jeff, The cloudera Hadoop is using guava-11.0.2.jar. I've also exported one environment variable before running pig: export HADOOP_USER_CLASSPATH_FIRST=true Should i recompile my PIG code with new guava jar? Regards, Sandeep On Wed, Sep 2, 2015 at 1:20 PM, Jianfeng (Jeff) Zhang <[email protected]<mailto:[email protected]>> wrote: It looks like classpath issue due guava version. Could you check the guava version CDH 5.4.4 is using ? AFAIK, Tez use guava 11.02 and pig use guava 11.0 Best Regard, Jeff Zhang From: Sandeep Kumar <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, September 2, 2015 at 2:28 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Pig(0.14.0) on Tez(0.7.0) Hi Guys, I've a pig script which is as follows: map = LOAD 'input/' using com.RawPigLoader('conf/Map.xml','conf/R360MapSignalling.json','csv'); normalized_map_data = foreach r360map generate flatten(com.Map($0..)); data_grouped_over_event_time = GROUP normalized_map_data by (((startTime/1000) / 3600) * 3600) PARTITION BY com.CustomTimePartitioner; final_data = foreach data_grouped_over_event_time generate flatten($1); store final_data into 'tmp/200MB' using com.StorageModel(); I'm trying to run my Pig job using Tez but i'm facing an exception which is attached. I searched over net and found some vague solutions. I've only two configuration entries in my tez-site.xml which are as follows: <property> <name>tez.lib.uris</name> <value>hdfs://node3:8020/user/hdfs/tez-0.7.0.tar.gz</value> </property> <property> <description>URL for where the Tez UI is hosted</description> <name>tez.tez-ui.history-url.base</name> <value>http://node4:9999/tez-ui/</value> </property> Is there anything else which i'm missing and supposed to be added in Tez configuration file. I've compiled PIG-0.14.0 manually with Tez-0.7.0 whereas it provided with Tez-0.5.2 but the exception is coming in both the versions of tez. Following are the version of softwares i'm using. HadoopVersion: 2.6.0-cdh5.4.4 PigVersion: 0.14.0-SNAPSHOT TezVersion: 0.7.0 Regards, Sandeep
