Malthe Borch created SPARK-35124:
------------------------------------
Summary: Local mode fails to start cluster due to configuration
value escape issue
Key: SPARK-35124
URL: https://issues.apache.org/jira/browse/SPARK-35124
Project: Spark
Issue Type: Bug
Components: Java API
Affects Versions: 3.1.1
Reporter: Malthe Borch
At least on Windows and perhaps also other systems, running Spark in local mode
fails to start a cluster when a configuration key contains a value with "&"
(ampersand) in it.
This happens during the "getOrCreate()" call from a Spark session builder.
The reason seems to be incorrect or insufficient escaping since on Windows, the
attempt to start a Spark process actually ends up running multiple commands for
each occurrence of the "&" character.
On Windows specifically, the correct way to escape "&" would be "^&", but I
have not been able yet to figure out exactly where the process is started and
how the configuration is passed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
