That's spark on YARN in Kerberos In Spark 1.3 you can submit work to a Kerberized Hadoop cluster; once the tokens you passed up with your app submission expire (~72 hours) your job can't access HDFS any more.
That's been addressed in Spark 1.4, where you can now specify a kerberos keytab for the application master; the AM will then give the workers updated tokens when needed. The kerberos authentication is all related to the HDFS interaction, YARN itself, and the way Kerberized YARN runs your work under your userid, not "mapred" or "yarn" It will also handle SPNEGO authentication between your web browser and the Spark UI (which is redirected via the YARN RM Proxy to achieve this) it does not do anything about Akka-based IPC between your client code and the spark application -steve On 11 Jun 2015, at 06:47, Akhil Das <ak...@sigmoidanalytics.com<mailto:ak...@sigmoidanalytics.com>> wrote: This might help http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.4/Apache_Spark_Quickstart_v224/content/ch_installing-kerb-spark-quickstart.html Thanks Best Regards On Wed, Jun 10, 2015 at 6:49 PM, kazeborja <kazebo...@gmail.com<mailto:kazebo...@gmail.com>> wrote: Hello all. I've been reading some old mails and notice that the use of kerberos in a standalone cluster was not supported. Is this stillt he case? Thanks. Borja. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-standalone-mode-and-kerberized-cluster-tp23255.html Sent from the Apache Spark User List mailing list archive at Nabble.com<http://Nabble.com>. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org<mailto:user-unsubscr...@spark.apache.org> For additional commands, e-mail: user-h...@spark.apache.org<mailto:user-h...@spark.apache.org>
