Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3559#issuecomment-66529795
Hey @brennonyork I'm actually not sure if we should introduce an additional
command line option to specify the environment file. I think it makes the
semantics very confusing because we can't be sure which environment file is
being used: is it the one specified through `--environment-file`? Is it the one
in `$SPARK_CONF_DIR/spark-env.sh`? Is it both? If it's both how to we merge the
values, or which file overwrites the other?
I think because of this circular dependency relationship between
`SPARK_CONF_DIR` and `spark-env.sh` we should just add a note in
`spark-env.sh.template` warning the user not to set `SPARK_CONF_DIR` there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
