Re: Can't find pyspark when using PySpark on YARN

2014-06-10 Thread Andrew Or
Hi Qi Ping, You don't have to distribute these files; they are automatically packaged in the assembly jar, which is already shipped to the worker nodes. Other people have run into the same issue. See if the instructions here are of any help: http://mail-archives.apache.org/mod_mbox/spark-user/201

Can't find pyspark when using PySpark on YARN

2014-06-10 Thread 李奇平
Dear all, When I submit a pyspark application using this command: ./bin/spark-submit --master yarn-client examples/src/main/python/wordcount.py "hdfs://..." I get the following exception: Error from python worker: Traceback (most recent call last): File "/usr/ali/lib/python2.5/runpy.py", line 85,