Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/33#discussion_r10152124
--- Diff: docs/security.md ---
@@ -0,0 +1,14 @@
+---
+layout: global
+title: Spark Security
+---
+
+Spark currently supports authentication via a shared secret.
Authentication can be configured to be on via the `spark.authenticate`
configuration parameter. This parameter controls whether the Spark
communication protocols do authentication using the shared secret. This
authentication is a basic handshake to make sure both sides have the same
shared secret and are allowed to communicate. If the shared secret is not
identical they will not be allowed to communicate.
+
+The Spark UI can also be secured by using javax servlet filters. A user
may want to secure the UI if it has data that other users should not be allowed
to see. The javax servlet filter specified by the user can authenticate the
user and then once the user is logged in, Spark can compare that user versus
the view acls to make sure they are authorized to view the UI. The configs
'spark.ui.acls.enable' and 'spark.ui.view.acls' control the behavior of the
acls. Note that the person who started the application always has view access
to the UI.
+
+For Spark on Yarn deployments, configuring `spark.authenticate` to true
will automatically handle generating and distributing the shared secret. Each
application will use a unique shared secret. The Spark UI uses the standard
YARN web application proxy mechanism and will authenticate via any installed
Hadoop filters. If an authentication filter is enabled, the acls controls can
be used by control which users can via the Spark UI.
+
+For other types of Spark deployments, the environment variable
`SPARK_SECRET` should be configured on each of the nodes. This secret will be
used by all the Master/Workers and applications. The UI can be secured using a
javax servlet filter installed via `spark.ui.filters`. If an authentication
filter is enabled, the acls controls can be used by control which users can via
the Spark UI.
+
+See [Spark Configuration](configuration.html) for more details on the
security configs.
--- End diff --
Here it might also be good to link on github to the javadoc of the
SecurityManager class (now that we have a stable home on github). Since it's so
thorough :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
