Yang Jie created SPARK-58872:
--------------------------------
Summary: Warn when driver credentials drop the driver service
account
Key: SPARK-58872
URL: https://issues.apache.org/jira/browse/SPARK-58872
Project: Spark
Issue Type: Improvement
Components: Kubernetes
Affects Versions: 5.0.0
Reporter: Yang Jie
`spark.kubernetes.authenticate.driver.serviceAccountName` is silently dropped
when the driver is also given its own client credentials.
`DriverKubernetesCredentialsFeatureStep.configurePod` applies the service
account only on the branch where there is no credentials secret to mount:
{code}
override def configurePod(pod: SparkPod): SparkPod = {
if (!shouldMountSecret) {
pod.copy(pod = buildPodWithServiceAccount(driverServiceAccount,
pod).getOrElse(pod.pod))
} else {
// mounts the credentials secret; driverServiceAccount is never read here
{code}
with
{code}
private val shouldMountSecret = oauthTokenBase64.isDefined ||
caCertDataBase64.isDefined ||
clientKeyDataBase64.isDefined ||
clientCertDataBase64.isDefined
{code}
This is the intended precedence, not a behavioral bug.
`docs/running-on-kubernetes.md` has said since 2.3.0 that the service account
"cannot be specified alongside a CA cert file, client key file, client cert
file, and/or OAuth token", and `Config.scala`'s doc for the entry says the
driver "will favor using those credentials instead". What is missing is any
enforcement or report: nothing rejects the combination and nothing tells the
user, so a submission that sets both is accepted and looks honored, while the
driver pod is left with the service account its spec already names, or the
namespace's `default`, and whatever RBAC that carries. The same doc page is
candid that the default account "may or may not have the role that allows
driver pods to create pods and services", so the mistake can surface much later
as an authorization failure when the driver requests executors.
`caCertFile` is the easiest of the four to trip over. It only establishes TLS
trust in the API server, so it is not by itself something the driver
authenticates with, yet `shouldMountSecret` treats it like a credential:
{code}
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.kubernetes.authenticate.driver.caCertFile=/path/to/ca.pem
{code}
drops the `spark` account. The driver then talks to the API server as the
namespace default account, whose in-pod token is what `Config.autoConfigure`
picks up.
Fix: log a warning naming the dropped account, the credential config(s) that
won, and `spark.kubernetes.authenticate.driver.mounted.*` as the way to keep
both. That prefix reads paths already present in the driver pod, never feeds
`shouldMountSecret`, and so leaves the service account in place. "Unset one of
the two" would be bad advice in both directions: unsetting `caCertFile` gives
up TLS trust in a private-CA API server, and unsetting the account also moves
executor pods off it, since `ExecutorKubernetesCredentialsFeatureStep` falls
back to the driver's account when
`spark.kubernetes.authenticate.executor.serviceAccountName` is unset.
A `require` that rejects the combination outright would match the documented
"cannot", but it would break submissions that are accepted today, so a warning
is the safer shape.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
