Nick, AFAIK, you can compile with yarn=true and still run spark in stand alone cluster mode. Cheers <k/>
On Wed, Jul 9, 2014 at 9:27 AM, Nick R. Katsipoulakis <kat...@cs.pitt.edu> wrote: > Hello, > > I am currently learning Apache Spark and I want to see how it integrates > with an existing Hadoop Cluster. > > My current Hadoop configuration is version 2.2.0 without Yarn. I have > build Apache Spark (v1.0.0) following the instructions in the README file. > Only setting the SPARK_HADOOP_VERSION=1.2.1. Also, I export the > HADOOP_CONF_DIR to point to the configuration directory of Hadoop > configuration. > > My use-case is the Linear Least Regression MLlib example of Apache Spark > (link: > http://spark.apache.org/docs/latest/mllib-linear-methods.html#linear-least-squares-lasso-and-ridge-regression). > The only difference in the code is that I give the text file to be an HDFS > file. > > However, I get a "Runtime Exception: Error in configuring object." > > So my question is the following: > > Does Spark work with a Hadoop distribution without Yarn? > If yes, am I doing it right? If no, can I build Spark with > SPARK_HADOOP_VERSION=2.2.0 and with SPARK_YARN=false? > > Thank you, > Nick >
