hadoop + yarn + spark

2014-06-27 Thread sdeb
Hello, I have installed spark on top of hadoop + yarn. when I launch the pyspark shell try to compute something I get this error. Error from python worker: /usr/bin/python: No module named pyspark The pyspark module should be there, do I have to put an external link to it? --Sanghamitra.

trouble: Launching spark on hadoop + yarn.

2014-06-25 Thread sdeb
i am trying to install spark on Hadoop+Yarn. I have installed spark using sbt (SPARK_HADOOP_VERSION=2.0.5-alpha SPARK_YARN=true sbt/sbt assembly ). This has worked fine. After that I am running : SPARK_JAR=./assembly/target/scala-2.10/spark-assembly-1.0.0-hadoop2.0.5-alpha.jar