[
https://issues.apache.org/jira/browse/SPARK-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-17256:
------------------------------------
Assignee: (was: Apache Spark)
> spark-submit.cmd cannot work if path has space and cut off double-quoted
> arguments
> ----------------------------------------------------------------------------------
>
> Key: SPARK-17256
> URL: https://issues.apache.org/jira/browse/SPARK-17256
> Project: Spark
> Issue Type: Bug
> Components: Deploy
> Affects Versions: 2.0.0
> Environment: Windows
> Reporter: Quanmao Liu
> Labels: cut-off, double-quotes
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> The key problem is { cmd /V /E /C "~%dp0spark-xxx.cmd" } cannot accept
> arguments has double quotes, even if itself of "spark-xxx.cmd".
> Double quoted command in spark/bin/spark-submit.cmd (and spark-class.cmd etc)
> as following:
> cmd /V /E /C "%~dp0spark-submit2.cmd" %*
> Will cut off the argument that double quoted, and not effect if full path of
> spark-submit.cmd has space.
> Example case :
> spark-submit.cmd --jars xxxx
> "jdbc:mysql://localhost:3306/lzdb?user=guest&password=abc123" my_table
> (1) Cannot just start spark-submit.cmd if it's full path has space, for
> example "D:\opengit\spark\bin - Copy\spark-submit.cmd" :
> D:\opengit\spark\bin - Copy>
> spark-submit.cmd --jars just-to-start
> "jdbc:mysql://localhost:3306/lzdb?user=guest&password=abc123" my_table
> ---- will get erorr :
> 'D:\opengit\spark\bin' is not recognized as an internal or external command,
> operable program or batch file.
> 'password' is not recognized as an internal or external command,
> operable program or batch file.
> (2) spark-submit.cmd can be started but will cut off arguments if path's no
> space . for example of above but no space:
> D:\opengit\spark\bin>
> spark-submit.cmd --jars just-to-start
> "jdbc:mysql://localhost:3306/lzdb?user=guest&password=abc123" my_table
> ----- will get error :
> 'password' is not recognized as an internal or external command,
> operable program or batch file.
> (1) A solution for this is just removing the "cmd /V /E /C" in the
> spark/bin/xxx.cmd files , but this will has possible "pollution" to the
> environment variables in the same cmd window (and later derived cmd window
> like started in it by "start cmd").
> But this will solve the space in path problem.
> (2) A mitigation is checking before adding double quotes, and added if really
> need. This just works well if path no space.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
