Zhenhua Xu created SPARK-17566:
----------------------------------
Summary: "--master yarn --deploy-mode cluster" gives "Launching
Python applications through spark-submit is currently only supported for local
files"
Key: SPARK-17566
URL: https://issues.apache.org/jira/browse/SPARK-17566
Project: Spark
Issue Type: Bug
Components: Spark Submit
Affects Versions: 2.0.0
Reporter: Zhenhua Xu
In Spark 1.6, the following command runs fine with both primary and additional
python files in hdfs.
/bin/spark-submit --py-files hdfs:///tmp/base.py --master yarn-cluster
hdfs:///tmp/pi.py
In Spark 2.0.0, the following command fails:
/bin/spark-submit --py-files hdfs:///tmp/base.py --master yarn --deploy-mode
cluster hdfs:///tmp/pi.py
Error:
Launching Python applications through spark-submit is currently only supported
for local files: hdfs:///tmp/base.py
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
