[
https://issues.apache.org/jira/browse/OOZIE-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Cseh updated OOZIE-3112:
------------------------------
Attachment: OOZIE-3112.06.patch
Improved logging and added more info about what's going on.
Output looks like this:
{code}
Reading Spark config propagated from Oozie server...
Reading Spark config from file spark-defaults.conf...
Value of defaultProperty3 was overwritten from spark-defaults.conf
Reading Spark config from --properties-file foo.properties...
Value of foo3 was overwritten from foo.properties
Value of defaultProperty2 was overwritten from foo.properties
Persisted merged Spark configs in file
spark-defaults-oozie-generated.properties. Merged properties are: [foo3,
defaultProperty3, foo2, defaultProperty2, defaultProperty, foo]
{code}
> SparkConfigrationService overwrites properties provided via --properties-file
> option in SparkAction
> ---------------------------------------------------------------------------------------------------
>
> Key: OOZIE-3112
> URL: https://issues.apache.org/jira/browse/OOZIE-3112
> Project: Oozie
> Issue Type: Bug
> Reporter: Peter Cseh
> Assignee: Peter Cseh
> Attachments: OOZIE-3112.01.patch, OOZIE-3112.02.patch,
> OOZIE-3112.03.patch, OOZIE-3112.04.patch, OOZIE-3112.05.patch,
> OOZIE-3112.06.patch
>
>
> SparkConfigurationService injects the config values as --conf to SparkSubmit.
> This will overwrite propties provided in the --properies-file option which is
> not the expected behavior.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
