Hi Kristoffer, You're correct that CDH5 only supports up to Java 7 at the moment. But Yarn apps do not run in the same JVM as Yarn itself (and I believe MR1 doesn't either), so it might be possible to pass arguments in a way that tells Yarn to launch the application master / executors with the Java 8 runtime. I have never tried this, so I don't know if it's really possible, and it's obviously not supported (also because Java 8 support is part of Spark 1.0 which hasn't been released yet).
You're welcome to try it out, and if you get it to work in some manner, it would be great to hear back. On Tue, May 6, 2014 at 6:16 AM, Kristoffer Sjögren <sto...@gmail.com> wrote: > Hi > > I just read an article [1] about Spark, CDH5 and Java 8 but did not get > exactly how Spark can run Java 8 on a YARN cluster at runtime. Is Spark > using a separate JVM that run on data nodes or is it reusing the YARN JVM > runtime somehow, like hadoop1? > > CDH5 only supports Java 7 [2] as far as I know? > > Cheers, > -Kristoffer > > > [1] > http://blog.cloudera.com/blog/2014/04/making-apache-spark-easier-to-use-in-java-with-java-8/ > [2] > http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Requirements-and-Supported-Versions/CDH5-Requirements-and-Supported-Versions.html > > > > -- Marcelo
