[
https://issues.apache.org/jira/browse/SPARK-16298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15366310#comment-15366310
]
Thomas Graves commented on SPARK-16298:
---------------------------------------
kinit as the user in the keytab before submitting the job.
I remember there were a few issues with this originally and I'm not sure if
they were all fixed. I thought if you weren't kinited yet it would use the
keytab to do so, but I'm not sure it supposed changing the user.
https://github.com/apache/spark/pull/9875 might be related.
If we change it we should make it consistent and always just login from keytab
or don't.
> spark.yarn.principal not working
> --------------------------------
>
> Key: SPARK-16298
> URL: https://issues.apache.org/jira/browse/SPARK-16298
> Project: Spark
> Issue Type: Bug
> Reporter: Partha Pratim Ghosh
>
> I am opening a Spark configuration with spark.yarn.principal and
> spark.yarn.keytab. However, this is not authenticating the underlying HDFS
> with the same principal and keytab. Instead, seems it is picking up from
> ticket cache. Without this feature the spark.yarn.principal and
> spark.yarn.keytab doesn't seem to be logical.
> Sample code -
> SparkConf conf = new
> SparkConf().setMaster("yarn-client").setAppName("spark-test")
> .set("spark.repl.class.uri",
> classServerUri);
> conf.set("spark.yarn.principal", principal);
> conf.set("spark.yarn.keytab", keytab);
> conf.setSparkHome(sparkBasePath);
>
> if (execUri != null) {
> conf.set("spark.executor.uri", execUri);
> }
> conf.set("spark.executor.memory", "8g");
> conf.set("spark.scheduler.mode", "FAIR");
> SparkContext sparkContext = new
> SparkContext(conf);
> Please advise how this can be achieved.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
