Github user nrchakradhar commented on a diff in the pull request:
https://github.com/apache/spark/pull/21669#discussion_r206447871
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala
---
@@ -107,7 +109,14 @@ private[spark] class Client(
def run(): Unit = {
val resolvedDriverSpec = builder.buildFromFeatures(kubernetesConf)
val configMapName = s"$kubernetesResourceNamePrefix-driver-conf-map"
- val configMap = buildConfigMap(configMapName,
resolvedDriverSpec.systemProperties)
+ val isKerberosEnabled =
kubernetesConf.getTokenManager.isSecurityEnabled
+ // HADOOP_SECURITY_AUTHENTICATION is defined as simple for the driver
and executors as
+ // they need only the delegation token to access secure HDFS, no need
to sign in to Kerberos
+ val maybeSimpleAuthentication =
+ if (isKerberosEnabled) Some((s"-D$HADOOP_SECURITY_AUTHENTICATION",
"simple")) else None
--- End diff --
This is our observation as well. Setting to "simple" is masking the actual
setting. When we are trying kerberos with spark thrift server we are facing
multiple issues and have stopped working on that for some time. We hope we can
resume it in some time. If there are patches to include, we can give it a try.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
