Github user gdearment commented on the issue:
https://github.com/apache/spark/pull/23174
The issue with requiring the use of secrets is quite a bit of work must be
done in order to secure a cluster to ensure that the secrets are themselves
secured. Most of the high level concerns are outlined
[here](https://kubernetes.io/docs/concepts/configuration/secret/#risks), but
you then need to understand the implications of this in the context of how
Spark is used. Its my understanding that Spark's implementation on Kubernetes
doesnât dictate how spark jobs should be run. As an example of where this can
quickly break down:
- Imagine a scenario where someone runs two drivers in the same namespace,
and rely on the default svc token to allow the driver to create its executors.
- The driver token will have GET access on the secret for its driver, but
also other secrets within the same namespace â such as the secret for driver
B.
- If the executors in job B can use the same service token as their driver,
then they have the ability of launching executors that can use the secret from
job A to connect an executor at which point you have a compromise.
There are lots of things that can be done to prevent this kind of attack
â run in different namespaces, donât allow executors to have k8s
permissions, etc., but it feels risky for spark to require the use of secrets
while leaving it an exercise for the reader to understand how to properly run
spark such that the secrets are actually secured.
At Palantir, we rely heavily on an external secret storage and
bootstrapping system, built around Vault, to securely pass secrets into pods.
Using external systems for these kinds of things, especially those built around
Vault, is a [common
practice](https://www.google.com/search?q=kubernetes+secrets+with+vault).
So really, what we'd like here is to avoid coupling the API for how spark
containers read secrets so that alternative implementations can be used that
some might consider more secure.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
