Github user rvesse commented on a diff in the pull request:
https://github.com/apache/spark/pull/21669#discussion_r207299324
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala
---
@@ -107,7 +109,14 @@ private[spark] class Client(
def run(): Unit = {
val resolvedDriverSpec = builder.buildFromFeatures(kubernetesConf)
val configMapName = s"$kubernetesResourceNamePrefix-driver-conf-map"
- val configMap = buildConfigMap(configMapName,
resolvedDriverSpec.systemProperties)
+ val isKerberosEnabled =
kubernetesConf.getTokenManager.isSecurityEnabled
+ // HADOOP_SECURITY_AUTHENTICATION is defined as simple for the driver
and executors as
+ // they need only the delegation token to access secure HDFS, no need
to sign in to Kerberos
+ val maybeSimpleAuthentication =
+ if (isKerberosEnabled) Some((s"-D$HADOOP_SECURITY_AUTHENTICATION",
"simple")) else None
--- End diff --
@ifilonenko Thanks, it does seem to be the case that this is required for
regular jobs but blocks STS (at least using the old Spark on K8S code base).
When we disabled it to enable use of STS in our internal fork it then broke
regular user jobs.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
