Jussi Jousimo created SPARK-18644:
-------------------------------------
Summary: spark-submit fails to run python scripts with specific
names
Key: SPARK-18644
URL: https://issues.apache.org/jira/browse/SPARK-18644
Project: Spark
Issue Type: Bug
Components: PySpark, Spark Submit
Affects Versions: 2.0.2
Environment: Ubuntu 16.04
Reporter: Jussi Jousimo
Priority: Minor
I'm trying to run simple python script named tokenize.py with spark-submit. The
script only imports SparkContext:
from pyspark import SparkContext
And I run it with:
spark-submit --packages org.apache.spark:spark-streaming-kafka-0-8_2.11:2.0.2
tokenize.py
However, the script fails:
ImportError: cannot import name SparkContext
I have set all necessary environment variables, etc. Strangely, it seems the
filename is causing this error. If I rename the file to, e.g., tokenizer.py and
run again, it runs fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
