Github user SevadaAbraamyan commented on a diff in the pull request:
https://github.com/apache/spark/pull/265#discussion_r13505208
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
---
@@ -75,6 +83,165 @@ class SparkHadoopUtil {
def getSecretKeyFromUserCredentials(key: String): Array[Byte] = { null }
+ /**
+ * Return whether Hadoop security is enabled or not.
+ *
+ * @return Whether Hadoop security is enabled or not
+ */
+ def isSecurityEnabled(): Boolean = {
+ UserGroupInformation.isSecurityEnabled
+ }
+
+ /**
+ * Do user authentication when Hadoop security is turned on. Used by the
driver.
+ *
+ * @param sc Spark context
+ */
+ def doUserAuthentication(sc: SparkContext) {
+ getAuthenticationType match {
+ case "keytab" => {
+ // Authentication through a Kerberos keytab file. Necessary for
+ // long-running services like Shark/Spark Streaming.
+ scheduleKerberosRenewTask(sc)
+ }
+ case _ => {
+ // No authentication needed. Assuming authentication is already
done
+ // before Spark is launched, e.g., the user has authenticated with
+ // Kerberos through kinit already.
+ // Renew a Hadoop delegation token and store the token into a file.
+ // Add the token file so it gets downloaded by every slave nodes.
+ sc.addFile(initDelegationToken().toString)
+ }
+ }
+ }
+
+ /**
+ * Get the user whom the task belongs to.
+ *
+ * @param userName Name of the user whom the task belongs to
+ * @return The user whom the task belongs to
+ */
+ def getTaskUser(userName: String): UserGroupInformation = {
+ val ugi = UserGroupInformation.createRemoteUser(userName)
+ // Change the authentication method to Kerberos
+ ugi.setAuthenticationMethod(
+ UserGroupInformation.AuthenticationMethod.KERBEROS)
+ // Get and add Hadoop delegation tokens for the user
+ val iter = getDelegationTokens().iterator()
+ while (iter.hasNext) {
+ ugi.addToken(iter.next())
+ }
+
+ ugi
+ }
+
+ /**
+ * Get the type of Hadoop security authentication.
+ *
+ * @return Type of Hadoop security authentication
+ */
+ private def getAuthenticationType: String = {
+ sparkConf.get("spark.hadoop.security.authentication")
--- End diff --
Should this not have a default value?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
