It was 4.2.0 - we've kept it this way for the last few releases for consistency.
On Thu, Nov 21, 2013 at 11:19 AM, Pierre Borckmans < [email protected]> wrote: > You are right about the typo. I did a copy paste from the website which > presents this mistake. > I forgot about this even though I submitted the correction to GutHub last > week ;) > > > > Message sent from a mobile service - excuse typos and abbreviations > > Le 21 nov. 2013 à 20:16, <[email protected]> a écrit : > > Thank u Pierre. I think u had typo, as it should be cdh and not chd > > > > Anyone know the prebuilt spark binaries for CDH4 which hadoop version was > built against? > > > > http://spark.incubator.apache.org/downloads.html > > > > was it 2.0.0-cdh4.4.0 or 2.0.0-mr1-cdh4.4.0 ? > > > > Thanks, > > Hussam > > *From:* Pierre Borckmans > [mailto:[email protected]<[email protected]>] > > *Sent:* Thursday, November 21, 2013 10:33 AM > *To:* [email protected] > *Subject:* Re: setting SPARK_HADOOP_VERSION > > > > Given your cdh4 distribution, you should use > > 2.0.0-mr1-chd4.4.0 > > That is if you don't want to use YARN > > Otherwise you should use: > 2.0.0-chd4.4.0 > > I think you were misreading the table as the column you mentioned is for > the HDP distribution. > > > > HTH > > > > Pierre > > > > Sent from a mobile, excuse typos and abbreviations > > Message sent from a mobile service - excuse typos and abbreviations > > > Le 21 nov. 2013 à 18:42, <[email protected]> a écrit : > > Hi, > > > > Can someone advise please on the following setting of SPARK_HADOOP_VERSION? > > > > I have spark 0.8 pre compiled binary of CDH4 downloaded from spark site, > running in CDH4 env. > > > > Running hadoop version shows > > > > [spark@kserv06 ~]$ hadoop version > > *Hadoop 2.0.0-cdh4.4.0* > > Subversion > file:///data/1/jenkins/workspace/generic-package-rhel64-6-0/topdir/BUILD/hadoop-2.0.0-cdh4.4.0/src/hadoop-common-project/hadoop-common-r > c0eba6cd38c984557e96a16ccd7356b7de835e79 > > Compiled by jenkins on Tue Sep 3 19:33:17 PDT 2013 > > From source with checksum ac7e170aa709b3ace13dc5f775487180 > > This command was run using > /opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-common-2.0.0-cdh4.4.0.jar > > [spark@kserv06 ~]$ > > > > So per checking > http://spark.incubator.apache.org/docs/latest/hadoop-third-party-distributions.html > > > > SPARK_HADOOP_VERSION should be 1.2.0 … is that correct? > > > > Not sure if it’s related but I am getting this exception when invoking > javaRDD.saveAsTextFile > > > > exception 'org.apache.hadoop.ipc.RemoteException: Server IPC version 7 > cannot communicate with client version 4' > > > > > > Thanks, > > Hussam > >
