[
https://issues.apache.org/jira/browse/OOZIE-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Cseh updated OOZIE-3112:
------------------------------
Attachment: OOZIE-3112.01.patch
Attaching patch. The precedence is in the properties-file:
- user-defined properites file via the --properties-file option
- localized spark-defaults.conf file in the container of the SparkAction
- Properties propagated via SparkConfigurationService.
This changes the earlier behavior. Earlier it was:
- if there was a --properties-file option and a spark-defaults.conf file, the
user-defined file was ignored
- properties propagated via SparkConfigurationService had precedence over the
--properties-file option or the localized spark-defaults.conf file
> SparkConfigrationService overwrites properties provided via --properties-file
> option in SparkAction
> ---------------------------------------------------------------------------------------------------
>
> Key: OOZIE-3112
> URL: https://issues.apache.org/jira/browse/OOZIE-3112
> Project: Oozie
> Issue Type: Bug
> Reporter: Peter Cseh
> Assignee: Peter Cseh
> Attachments: OOZIE-3112.01.patch
>
>
> SparkConfigurationService injects the config values as --conf to SparkSubmit.
> This will overwrite propties provided in the --properies-file option which is
> not the expected behavior.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
