Github user harishreedharan commented on a diff in the pull request:
https://github.com/apache/spark/pull/4688#discussion_r27252140
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -540,6 +563,26 @@ private[spark] class Client(
amContainer
}
+ def setupCredentials(): Unit = {
+ if (args.principal != null) {
+ require(args.keytab != null, "Keytab must be specified when
principal is specified.")
+ logInfo("Attempting to login to the Kerberos" +
+ s" using principal: ${args.principal} and keytab: ${args.keytab}")
+ val f = new File(args.keytab)
+ // Generate a file name that can be used for the keytab file, that
does not conflict
+ // with any user file.
+ val keytabFileName = f.getName + "-" + UUID.randomUUID().toString
+ val ugi =
UserGroupInformation.loginUserFromKeytabAndReturnUGI(args.principal,
args.keytab)
+ credentials = ugi.getCredentials
+ loginFromKeytab = true
+ sparkConf.set("spark.yarn.keytab", keytabFileName)
--- End diff --
The main reason I kept this as command line args is that it makes it easy
for a user to use the same config file for different apps which may use
different credentials. I have seen some customers do this, and in some cases
may want to use different credentials to write to HDFS from different apps(and
don't want proxying). I wouldn't mind supporting both, with the command line
params overriding the conf values, but I'd rather do that in a separate PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
