paul mackles created SPARK-24380:
------------------------------------
Summary: argument quoting/escaping broken
Key: SPARK-24380
URL: https://issues.apache.org/jira/browse/SPARK-24380
Project: Spark
Issue Type: Bug
Components: Deploy, Mesos
Affects Versions: 2.3.0, 2.2.0
Reporter: paul mackles
Fix For: 2.4.0
When a configuration property contains shell characters that require quoting,
the Mesos cluster scheduler generates the spark-submit argument like so:
{code:java}
--conf "spark.mesos.executor.docker.parameters="label=logging=|foo|""{code}
Note the quotes around the property value as well as the key=value pair. When
using docker, this breaks the spark-submit command and causes the "|" to be
interpreted as an actual shell PIPE. Spaces, semi-colons, etc also cause issues.
Although I haven't tried, I suspect this is also a potential security issue in
that someone could exploit it to run arbitrary code on the host.
My patch is pretty minimal and just removes the outer quotes around the
key=value pair, resulting in something like:
{code:java}
--conf spark.mesos.executor.docker.parameters="label=logging=|foo|"{code}
A more extensive fix might try wrapping the entire key=value pair in single
quotes but I was concerned about backwards compatibility with that change.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
