[
https://issues.apache.org/jira/browse/SPARK-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Wendell updated SPARK-2664:
-----------------------------------
Description:
If someone sets a spark conf that relates to an existing flag `--master`, we
should set it correctly like we do with the defaults file. Otherwise it can
have confusing semantics. I noticed this after merging it, otherwise I would
have mentioned it in the review.
I think it's as simple as modifying loadDefaults to check the user-supplied
options also. We might change it to loadUserProperties since it's no longer
just the defaults file.
was:If someone sets a spark conf that relates to an existing flag `--master`,
we should set it correctly like we do with the defaults file. Otherwise it can
have confusing semantics. I noticed this after merging it, otherwise I would
have mentioned it in the review.
> Deal with `--conf` options in spark-submit that relate to flags
> ---------------------------------------------------------------
>
> Key: SPARK-2664
> URL: https://issues.apache.org/jira/browse/SPARK-2664
> Project: Spark
> Issue Type: Bug
> Reporter: Patrick Wendell
> Assignee: Sandy Ryza
> Priority: Blocker
>
> If someone sets a spark conf that relates to an existing flag `--master`, we
> should set it correctly like we do with the defaults file. Otherwise it can
> have confusing semantics. I noticed this after merging it, otherwise I would
> have mentioned it in the review.
> I think it's as simple as modifying loadDefaults to check the user-supplied
> options also. We might change it to loadUserProperties since it's no longer
> just the defaults file.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
