Given your cdh4 distribution, you should use 2.0.0-mr1-chd4.4.0
That is if you don't want to use YARN Otherwise you should use: 2.0.0-chd4.4.0 I think you were misreading the table as the column you mentioned is for the HDP distribution. HTH Pierre Sent from a mobile, excuse typos and abbreviations Message sent from a mobile service - excuse typos and abbreviations > Le 21 nov. 2013 à 18:42, <[email protected]> a écrit : > > Hi, > > Can someone advise please on the following setting of SPARK_HADOOP_VERSION? > > I have spark 0.8 pre compiled binary of CDH4 downloaded from spark site, > running in CDH4 env. > > Running hadoop version shows > > [spark@kserv06 ~]$ hadoop version > Hadoop 2.0.0-cdh4.4.0 > Subversion > file:///data/1/jenkins/workspace/generic-package-rhel64-6-0/topdir/BUILD/hadoop-2.0.0-cdh4.4.0/src/hadoop-common-project/hadoop-common > -r c0eba6cd38c984557e96a16ccd7356b7de835e79 > Compiled by jenkins on Tue Sep 3 19:33:17 PDT 2013 > From source with checksum ac7e170aa709b3ace13dc5f775487180 > This command was run using > /opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-common-2.0.0-cdh4.4.0.jar > [spark@kserv06 ~]$ > > So per checking > http://spark.incubator.apache.org/docs/latest/hadoop-third-party-distributions.html > > SPARK_HADOOP_VERSION should be 1.2.0 … is that correct? > > Not sure if it’s related but I am getting this exception when invoking > javaRDD.saveAsTextFile > > exception 'org.apache.hadoop.ipc.RemoteException: Server IPC version 7 cannot > communicate with client version 4' > > > Thanks, > Hussam
