[ https://issues.apache.org/jira/browse/SPARK-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15865434#comment-15865434 ]
Saisai Shao commented on SPARK-19588: ------------------------------------- Putting on HDFS still requires downloading to local disk for driver/yarn#client, since driver/yarn#client is not in control by yarn, so there's no difference whether putting it locally or on HDFS. > Allow putting keytab file to HDFS location specified in spark.yarn.keytab > ------------------------------------------------------------------------- > > Key: SPARK-19588 > URL: https://issues.apache.org/jira/browse/SPARK-19588 > Project: Spark > Issue Type: New Feature > Components: Spark Core, Spark Submit > Affects Versions: 2.0.2, 2.1.0 > Environment: kerberized cluster, Spark 2 > Reporter: Ruslan Dautkhanov > Labels: authentication, kerberos, security, yarn-client > > As a workaround for SPARK-19038 tried putting keytab in user's home directory > in HDFS but this fails with > {noformat} > Exception in thread "main" org.apache.spark.SparkException: Keytab file: > hdfs:///user/svc_odiprd/.kt does not exist > at > org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:555) > at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:158) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > {noformat} > This is yarn-client mode, so driver probably can't see HDFS while submitting > a job; although I suspect it doesn't not only with yarn-client. > Would be great to support reading keytab for kerberos ticket renewals > directly from HDFS. > We think that in some scenarios it's more secure than referencing a keytab > from a local fs on a client machine that does a spark-submit. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org