Hello,
I have installed spark on top of hadoop + yarn.
when I launch the pyspark shell try to compute something I get this error.
Error from python worker:
/usr/bin/python: No module named pyspark
The pyspark module should be there, do I have to put an external link to it?
--Sanghamitra.
i am trying to install spark on Hadoop+Yarn.
I have installed spark using sbt (SPARK_HADOOP_VERSION=2.0.5-alpha
SPARK_YARN=true sbt/sbt assembly ). This has worked fine.
After that I am running :
SPARK_JAR=./assembly/target/scala-2.10/spark-assembly-1.0.0-hadoop2.0.5-alpha.jar