Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16827
@srowen Well, this change may be really radical. Just like what I say, many
fresh users has no consciousness to check if job runs correctly in standalone
mode or yarn mode. Maybe, they just forgot to remove the "local[*]" from code
after debug or just did not know the priority of setting 'spark.master'. This
is not a rare case. I have have received many reports like this. Indeed, we can
say only newbie may make this mistake. But it is better to warn users in some
ways.
What about just log a warning? But, I don't think users will notice it.
Besides, I do not find any strong reason to set 'spark.master' multi-times
with different value.
Any suggestion is appreciated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
