Pierre Beauvois created SPARK-13110:
---------------------------------------
Summary: How to configure the access of the Spark History Web UI
with Kerberos authentication?
Key: SPARK-13110
URL: https://issues.apache.org/jira/browse/SPARK-13110
Project: Spark
Issue Type: Question
Components: Web UI
Affects Versions: 1.6.0, 1.5.2, 1.5.1
Environment: Spark 1.6.0 / Hadoop 2.7.1 / Zookeeper 3.4.5 /
Authentication done through Kerberos
Reporter: Pierre Beauvois
Hello,
Spark is installed on several machines of my cluster. These machines are used
by the clients (we'll call these machines "CM"). Note that Spark is configured
on Yarn and not on standalone mode.
I installed a Spark History server on a different machine which is unaccessible
to the clients (we'll call this machine "SHS"). Now I'm trying to configure
Spark History web UI access for the users of my cluster who are authenticated
with Kerberos. For the moment I have been able to make Spark History working
with its kerberos principal and keytab.
The spark-defaults.conf of the SHS is the following:
{code}
# Spark history server configurations
spark.history.provider = org.apache.spark.deploy.history.FsHistoryProvider
spark.history.fs.logDirectory = hdfs:///Products/SPARK/logs/
spark.history.fs.update.interval = 10s
spark.history.retainedApplications = 100
spark.history.ui.port = 18080
spark.history.kerberos.enabled = true
spark.history.kerberos.principal =
sparkhistory/[email protected]
spark.history.kerberos.keytab =
/opt/application/Spark/current/keytabs/sparkhistory.keytab
spark.history.ui.acls.enable = true
spark.history.fs.cleaner.enabled = false
spark.yarn.historyServer.address = sparkhistoryserver.dns.fr:18080
{code}
The spark-defaults.conf of the CM is the following:
{code}
# Spark history server configurations
spark.history.provider =
spark.history.fs.logDirectory =
spark.history.fs.update.interval =
spark.history.retainedApplications =
spark.history.ui.port =
spark.history.kerberos.enabled =
spark.history.kerberos.principal =
spark.history.kerberos.keytab =
spark.history.ui.acls.enable = true
spark.history.fs.cleaner.enabled =
spark.yarn.historyServer.address = sparkhistoryserver.dns.fr:18080
{code}
First I would like to know if my configurations are good.
Secondly I would like to know how to restrict the web UI access for the users
who are kerberos authenticated. Let me explain more what is the expected
behaviour here:
- the user Obelix does a Spark job and finishes it properly
- Obelix can go to the ResourceManager web UI and click on "history". He's
redirected to the Spark History web UI and he can have the details of its
previous job. Note that Obelix is kerberos authenticated in order to be able to
go to the ResourceManager web UI and the Spark History web UI.
- Asterix goes to the ResourceManager web UI and click on "history" of Obelix's
job. Asterix is not redirected because he's not the user who launched the job.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
