Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/1233#issuecomment-47252173
Hi @witgo, can you elaborate, in the change summary, why this is needed?
Maybe file a bug?
I think you guys don't use spark-submit, which is why you might need this;
but spark-submit translates the config file into system properties, so the
config is picked up by SparkConf. And it seems like this change is breaking the
semantics of how spark-submit works.
(I think it would be nice to stop using system properties like that - it
gets really confusing when parts of the code use system properties and others
use SparkConf - but that's a separate discussion.)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
