Hi, Can someone advise please on the following setting of SPARK_HADOOP_VERSION?
I have spark 0.8 pre compiled binary of CDH4 downloaded from spark site, running in CDH4 env. Running hadoop version shows [spark@kserv06 ~]$ hadoop version Hadoop 2.0.0-cdh4.4.0 Subversion file:///data/1/jenkins/workspace/generic-package-rhel64-6-0/topdir/BUILD/hadoop-2.0.0-cdh4.4.0/src/hadoop-common-project/hadoop-common -r c0eba6cd38c984557e96a16ccd7356b7de835e79 Compiled by jenkins on Tue Sep 3 19:33:17 PDT 2013 >From source with checksum ac7e170aa709b3ace13dc5f775487180 This command was run using /opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-common-2.0.0-cdh4.4.0.jar [spark@kserv06 ~]$ So per checking http://spark.incubator.apache.org/docs/latest/hadoop-third-party-distributions.html SPARK_HADOOP_VERSION should be 1.2.0 ... is that correct? Not sure if it's related but I am getting this exception when invoking javaRDD.saveAsTextFile exception 'org.apache.hadoop.ipc.RemoteException: Server IPC version 7 cannot communicate with client version 4' Thanks, Hussam
