Vincent Sun created SPARK-20016:
-----------------------------------
Summary: SparkLauncher submit job failed when setConf with special
charaters under windows
Key: SPARK-20016
URL: https://issues.apache.org/jira/browse/SPARK-20016
Project: Spark
Issue Type: Bug
Components: Java API
Affects Versions: 2.0.0
Environment: windows 7, 8, 10, 2008, 2008R2, etc.
Reporter: Vincent Sun
I am using sparkLauncher JAVA API to submit job to a remote spark cluster
master. I have some configurations like --conf spark.executor.cores=2 need to
pass to master. So I called SparkLauncher.setConf() method. It works fine under
Linux/CentOS, but failed on my own desktop which is a windows 8 OS. After debug
into the source code I found the reason is at:
quoteForBatchScript method of CommandBuilderUtils class
It will add quote '"' characters while there is '=' or some other kinds of
special characters under windows system. And that cause the command cannot be
executed success in windows cmd.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
