Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3571#discussion_r23964514
--- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala ---
@@ -142,9 +148,39 @@ import org.apache.spark.network.sasl.SecretKeyHolder
* authentication. Spark will then use that user to compare against the
view acls to do
* authorization. If not filter is in place the user is generally null
and no authorization
* can take place.
+ *
+ * Connection encryption (SSL) configuration is organized hierarchically.
The user can configure
+ * the default SSL settings which will be used for all the supported
communication protocols unless
+ * they are overwritten by protocol specific settings. This way the user
can easily provide the
+ * common settings for all the protocols without disabling the ability to
configure each one
+ * individually.
+ *
+ * All the SSL settings like `spark.ssl.xxx` where `xxx` is a particular
configuration property,
+ * denote the global configuration for all the supported protocols. In
order to override the global
+ * configuration for the particular protocol, the properties must be
overwritten in the
+ * protocol-specific namespace. Use `spark.ssl.yyy.xxx` settings to
overwrite the global
+ * configuration for particular protocol denoted by `yyy`. Currently
`yyy` can be either `akka` for
+ * Akka based connections or `fs` for broadcast and file server.
+ *
+ * Refer to [[org.apache.spark.SSLOptions]] documentation for the list of
+ * options that can be specified.
+ *
+ * SecurityManager initializes SSLOptions objects for different protocols
separately. SSLOptions
+ * object parses Spark configuration at a given namespace and builds the
common representation
+ * of SSL settings. SSLOptions is the used to provide protocol-specific
configuration like TypeSafe
+ * configuration for Akka or SSLContextFactory for Jetty.
+ * SSL must be configured on each node and configured for each component
involved in
+ * communication using the particular protocol. In YARN clusters, the
key-store can be prepared on
+ * the client side then distributed and used by the executors as the part
of the application
+ * (YARN allows the user to deploy files before the application is
started).
+ * In standalone deployment, the user needs to provide key-stores and
configuration
+ * options for master and workers. In this mode, the user may allow the
executors to use the SSL
--- End diff --
I realize this is kind of pedantic, but is the difference between YARN and
Standalone that standalone users have to handle key _distribution_ themselves
whereas YARN handles it automatically? If I understand correctly, the user is
still responsible for generating the key-stores in both modes and the only
difference is whether there's a secure distribution mechanism for those stores.
If this is the case, maybe we can state this a bit more explicitly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
