[
https://issues.apache.org/jira/browse/OOZIE-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099681#comment-16099681
]
Andras Piros commented on OOZIE-2984:
-------------------------------------
[~gezapeti] please see {{TestSparkOptionsSplitter}} test cases for details, an
example test case that covers the use case when there are white spaces in the
option values:
{noformat}
--conf special1=value1 value2 special2=value3 value4
{noformat}
Other test cases cover the existing functionality within the same test class,
and there are yet another ones when we have quotes inside the values (but not
all the option value is quoted).
[~pbacsko] addressing your comments.
> Parse spark-defaults.conf values with spaces without needing the quotes
> -----------------------------------------------------------------------
>
> Key: OOZIE-2984
> URL: https://issues.apache.org/jira/browse/OOZIE-2984
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 4.3.0
> Reporter: Fahd Siddiqui
> Assignee: Andras Piros
> Fix For: 5.0.0
>
> Attachments: OOZIE-2984.001.patch
>
>
> Oozie requires multiple java opts to be enclosed in quotes. For example:
> {code}
> spark.driver.extraJavaOptions="-Xmn2703m -XX:SurvivorRatio=2
> -XX:ParallelGCThreads=20"
> {code}
> However the above breaks spark-shell.
> {code}
> $ ./spark2-shell
> Invalid initial young generation size: -Xmn2212m -XX:SurvivorRatio=2
> -XX:ParallelGCThreads=20
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> {code}
> To fix spark-shell, we have to remove the quotes, which errors out Oozie:
> {code}
> Error: Unrecognized option '-XX:SurvivorRatio=2'
> ...
> --conf
> spark.executor.extraJavaOptions=-Xmn2703m
> -Dlog4j.configuration=spark-log4j.properties
> -XX:SurvivorRatio=2
> {code}
> Oozie should be able to parse spark-defaults.conf values with spaces without
> needing the quotes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
