Leighton Wong created SPARK-18114:
-------------------------------------
Summary: MesosClusterScheduler generate bad command options
Key: SPARK-18114
URL: https://issues.apache.org/jira/browse/SPARK-18114
Project: Spark
Issue Type: Bug
Components: Mesos
Affects Versions: 2.0.1
Environment: CentOS 7, Jdk8, scala 2.11.8
Reporter: Leighton Wong
Fix For: 2.0.2
{code:title=MesosClusterScheduler.java|borderStyle=solid}
desc.schedulerProperties
.filter { case (key, _) => !replicatedOptionsBlacklist.contains(key) }
.foreach { case (key, value) => options ++= Seq("--conf",
s"$key=${shellEscape(value)}") }
options
{code}
As above, --conf option value is not enclosed by "", for example, if we have a
multi value config like
{quote}
spark.driver.extraJavaOptions=
-Dlog4j.configuration=file:./conf/log4j-server.properties
-Dalluxio.user.file.writetype.default=THROUGH
{quote}
Without "" the next alluxio config will be treated as spark-submit options and
cause error.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
