[jira] [Commented] (SPARK-2914) spark.*.extraJavaOptions are evaluated too many times
[
https://issues.apache.org/jira/browse/SPARK-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14310131#comment-14310131
]
Andrew Or commented on SPARK-2914:
--
I think these two describe the same issues, but the description in 2722 is a
little clearer. I will close this in favor of that one.
spark.*.extraJavaOptions are evaluated too many times
-
Key: SPARK-2914
URL: https://issues.apache.org/jira/browse/SPARK-2914
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.0.2
Reporter: Andrew Or
Assignee: Andrew Or
Priority: Critical
If we pass the following to spark.executor.extraJavaOptions,
{code}
-Dthem.quotes=the \best\ joke ever -Dthem.backslashes= \\ \\
{code}
These will first be escaped once when the SparkSubmit JVM is launched. This
becomes the following string.
{code}
scala sc.getConf.get(spark.driver.extraJavaOptions)
res0: String = -Dthem.quotes=the best joke ever -Dthem.backslashes= \ \
\\
{code}
This will be split incorrectly by Utils.splitCommandString.
Of course, this also affects spark.driver.extraJavaOptions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2914) spark.*.extraJavaOptions are evaluated too many times
[
https://issues.apache.org/jira/browse/SPARK-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259463#comment-14259463
]
Sean Owen commented on SPARK-2914:
--
Is this a superset of SPARK-2722 or vice versa?
spark.*.extraJavaOptions are evaluated too many times
-
Key: SPARK-2914
URL: https://issues.apache.org/jira/browse/SPARK-2914
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.0.2
Reporter: Andrew Or
Assignee: Andrew Or
Priority: Critical
Fix For: 1.2.0
If we pass the following to spark.executor.extraJavaOptions,
{code}
-Dthem.quotes=the \best\ joke ever -Dthem.backslashes= \\ \\
{code}
These will first be escaped once when the SparkSubmit JVM is launched. This
becomes the following string.
{code}
scala sc.getConf.get(spark.driver.extraJavaOptions)
res0: String = -Dthem.quotes=the best joke ever -Dthem.backslashes= \ \
\\
{code}
This will be split incorrectly by Utils.splitCommandString.
Of course, this also affects spark.driver.extraJavaOptions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2914) spark.*.extraJavaOptions are evaluated too many times
[
https://issues.apache.org/jira/browse/SPARK-2914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14090237#comment-14090237
]
Andrew Or commented on SPARK-2914:
--
https://github.com/apache/spark/pull/1845
spark.*.extraJavaOptions are evaluated too many times
-
Key: SPARK-2914
URL: https://issues.apache.org/jira/browse/SPARK-2914
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.0.2
Reporter: Andrew Or
Fix For: 1.1.0
If we pass the following to spark.executor.extraJavaOptions,
{code}
-Dthem.quotes=the \best\ joke ever -Dthem.backslashes= \\ \\
{code}
These will first be escaped once when the SparkSubmit JVM is launched. This
becomes the following string.
{code}
scala sc.getConf.get(spark.driver.extraJavaOptions)
res0: String = -Dthem.quotes=the best joke ever -Dthem.backslashes= \ \
\\
{code}
This will be split incorrectly by Utils.splitCommandString.
Of course, this also affects spark.driver.extraJavaOptions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
