Github user skonto commented on a diff in the pull request:
https://github.com/apache/spark/pull/20945#discussion_r178820771
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala
---
@@ -506,6 +506,10 @@ private[spark] class MesosClusterScheduler(
options ++= Seq("--class", desc.command.mainClass)
}
+ desc.conf.getOption("spark.mesos.proxyUser").foreach { v =>
+ options ++= Seq("--proxy-user", v)
--- End diff --
@vanzin
On dc/os the spark dcos cli which supports kerberos & keytab paths submits
jobs directly to the
mesos rest api at the mesos dispatcher side. The keytabs are stored on the
DC/OS secret store before the job is launched and they are mounted on the
container before container is launched.
The idea here is to store the keytab for the superuser on the secret store,
so the spark driver which is eventually launched in client mode within the
cluster, to login to kerberos and impersonate another user. The driver will
start the the SparkJob's main as a proxy user (as usual) and will use the
superuser credentials to impersonate the passed proxy user.
The OS used by the container depends on the setup but that should be with
minimal rights.
Right now DC/OS switched back to root, previously it used nobody but users
can customize the image to add their users anywy. In containerized envs this
add extra headaches.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
