HI guys
I have documented the steps involved in getting Spark 1.5.1 run on CDH
5.4.0 here, let me know if it works for you as well
https://www.linkedin.com/pulse/running-spark-151-cdh-deenar-toraskar-cfa?trk=hp-feed-article-title-publish
Looking forward to CDH 5.5 which supports Spark 1.5.x out o
I got this working. For others trying this It turns out in Spark 1.3/CDH5.4
spark.yarn.jar=local:/opt/cloudera/parcels/
I had changed this to reflect the 1.5.1 version of spark assembly jar
spark.yarn.jar=/opt/spark-1.5.1-bin/...
and this didn't work, I had to drop the "local:" prefix
spar
Sandy
The assembly jar does contain org.apache.spark.deploy.yarn.ExecutorLauncher.
I am trying to find out how i can increase the logging level, so I know the
exact classpath used by Yarn ContainerLaunch.
Deenar
On 23 October 2015 at 03:30, Sandy Ryza wrote:
> Hi Deenar,
>
> The version of Spa
Hi Deenar,
The version of Spark you have may not be compiled with YARN support. If
you inspect the contents of the assembly jar, does
org.apache.spark.deploy.yarn.ExecutorLauncher exist? If not, you'll need
to find a version that does have the YARN classes. You can also build your
own using the
Hi I have got the prebuilt version of Spark 1.5 for Hadoop 2.6 (
http://www.apache.org/dyn/closer.lua/spark/spark-1.5.1/spark-1.5.1-bin-hadoop2.6.tgz)
working with CDH 5.4.0 in local mode on a cluster with Kerberos. It works
well including connecting to the Hive metastore. I am facing an issue
runn