[
https://issues.apache.org/jira/browse/OOZIE-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16098391#comment-16098391
]
Peter Cseh commented on OOZIE-2984:
-----------------------------------
The original issue was that when spark-default.conf contained a line with
whitespaces in the value, SparkMain parsed it incorrectly. I don't know if it's
possible to test this out in JUnit environment but I don't think this patch
solves that issue. It only allows to have escapes quotes inside quoted
properties.
> Parse spark-defaults.conf values with spaces without needing the quotes
> -----------------------------------------------------------------------
>
> Key: OOZIE-2984
> URL: https://issues.apache.org/jira/browse/OOZIE-2984
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 4.3.0
> Reporter: Fahd Siddiqui
> Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-2984.001.patch
>
>
> Oozie requires multiple java opts to be enclosed in quotes. For example:
> {code}
> spark.driver.extraJavaOptions="-Xmn2703m -XX:SurvivorRatio=2
> -XX:ParallelGCThreads=20"
> {code}
> However the above breaks spark-shell.
> {code}
> $ ./spark2-shell
> Invalid initial young generation size: -Xmn2212m -XX:SurvivorRatio=2
> -XX:ParallelGCThreads=20
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> {code}
> To fix spark-shell, we have to remove the quotes, which errors out Oozie:
> {code}
> Error: Unrecognized option '-XX:SurvivorRatio=2'
> ...
> --conf
> spark.executor.extraJavaOptions=-Xmn2703m
> -Dlog4j.configuration=spark-log4j.properties
> -XX:SurvivorRatio=2
> {code}
> Oozie should be able to parse spark-defaults.conf values with spaces without
> needing the quotes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
