[
https://issues.apache.org/jira/browse/OOZIE-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15588771#comment-15588771
]
Abhishek Bafna commented on OOZIE-2712:
---------------------------------------
{code}
propertiesToMerge.put(DRIVER_CLASSPATH_OPTION, DRIVER_CLASSPATH);
propertiesToMerge.put(DRIVER_MEMORY_OPTION, DRIVER_MEMORY);
propertiesToMerge.put(DRIVER_LIBRARY_PATH_OPTION,
DRIVER_EXTRA_LIBRARY_PATH);
propertiesToMerge.put(NUM_EXECUTOR_OPTION, EXECUTOR_INSTANCES);
propertiesToMerge.put(DRIVER_JAVA_OPTIONS_OPTION,
DRIVER_EXTRA_JAVA_OPTIONS);
propertySeparators.put(DRIVER_CLASSPATH, ",");
propertySeparators.put(DRIVER_EXTRA_LIBRARY_PATH, ",");
propertySeparators.put(DRIVER_EXTRA_JAVA_OPTIONS, "");
propertySeparators.put(EXECUTOR_INSTANCES, "");
propertySeparators.put(DRIVER_EXTRA_JAVA_OPTIONS, " ");
{code}
{{DRIVER_EXTRA_JAVA_OPTIONS}} is added twice into {{propertySeparators}} which
is why {{DRIVER_MEMORY}} is missed out from the map.
> Merge Spark opts with two possible options
> -------------------------------------------
>
> Key: OOZIE-2712
> URL: https://issues.apache.org/jira/browse/OOZIE-2712
> Project: Oozie
> Issue Type: Bug
> Reporter: Peter Cseh
> Assignee: Peter Cseh
> Attachments: OOZIE-2712.001.patch
>
>
> Issues like OOZIE-2658 can happen with more properties.
> || --conf || overwritten by ||
> |spark.driver.memory | --driver-memory |
> |spark.driver.extraJavaOptions | --driver-java-options|
> |spark.driver.extraLibraryPath|--driver-library-path |
> |spark.executor.instances|--num-executors|
> |spark.driver.extraClassPath| --driver-class-path|
> We should make sure that one is converted to the other (the current pattern
> is to use --conf versions of these) so we don't run into issues where the
> log4j properties are overwritten. (OOZIE-2705 can be affected by this)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
