Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/9287#issuecomment-154148874
> But the thing about env variable needs explanation - actually the user is
supposed to provide a secret key in env variable.
I haven't finished my review yet, but I was going to get to that. That
change is really not necessary.
For the secret needed to connect to the Spark Master, that can be in the
configuration file. Everybody needs to share the same secret, so there's no
problem in it being there. Until the Master can do proper authentication, we'll
have to live with that fact.
For the per-app secret, it should be auto-generated, like in YARN. The user
should not need to provide their own secret every time they launch their app.
So you don't really have to set any env variables. The env variable is just
a way to get the secret from the Worker to the executor without using the
command line.
BTW, I think this is a step in the right direction, but because of the
above, this is not really adding any extra security. Because all Spark-launched
processes will run as the same user, it's possible for any malicious executor
to just read other app's secrets from the Worker's memory (or directly from
other running executors' environment in /proc).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
