Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/20742#discussion_r173609465
--- Diff: docs/security.md ---
@@ -58,49 +301,204 @@ component-specific configuration namespaces used to
override the default setting
</tr>
</table>
-The full breakdown of available SSL options can be found on the
[configuration page](configuration.html).
-SSL must be configured on each node and configured for each component
involved in communication using the particular protocol.
+The full breakdown of available SSL options can be found below. The
`${ns}` placeholder should be
+replaced with one of the above namespaces.
+
+<table class="table">
+<tr><th>Property Name</th><th>Default</th><th>Meaning</th></tr>
+ <tr>
+ <td><code>${ns}.enabled</code></td>
+ <td>false</td>
+ <td>Enables SSL. When enabled, <code>${ns}.ssl.protocol</code> is
required.</td>
+ </tr>
+ <tr>
+ <td><code>${ns}.port</code></td>
+ <td>None</td>
+ <td>
+ The port where the SSL service will listen on.
+
+ <br />The port must be defined within a specific namespace
configuration. The default
+ namespace is ignored when reading this configuration.
+
+ <br />When not set, the SSL port will be derived from the non-SSL
port for the
+ same service. A value of "0" will make the service bind to an
ephemeral port.
+ </td>
+ </tr>
+ <tr>
+ <td><code>${ns}.enabledAlgorithms</code></td>
+ <td>None</td>
+ <td>
+ A comma separated list of ciphers. The specified ciphers must be
supported by JVM.
+
+ <br />The reference list of protocols can be found in the "JSSE
Cipher Suite Names" section
+ of the Java security guide. The list for Java 8 can be found at
+ <a
href="https://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html#ciphersuites";>this</a>
+ page.
+
+ <br />Note: If not set, the default cipher suite for the JRE will be
used.
+ </td>
+ </tr>
+ <tr>
+ <td><code>${ns}.keyPassword</code></td>
+ <td>None</td>
+ <td>
+ The password to the private key in the key store.
+ </td>
+ </tr>
+ <tr>
+ <td><code>${ns}.keyStore</code></td>
+ <td>None</td>
+ <td>
+ Path to the key store file. The path can be absolute or relative to
the directory in which the
+ process is started.
+ </td>
+ </tr>
+ <tr>
+ <td><code>${ns}.keyStorePassword</code></td>
+ <td>None</td>
+ <td>Password to the key store.</td>
+ </tr>
+ <tr>
+ <td><code>${ns}.keyStoreType</code></td>
+ <td>JKS</td>
+ <td>The type of the key store.</td>
+ </tr>
+ <tr>
+ <td><code>${ns}.protocol</code></td>
+ <td>None</td>
+ <td>
+ TLS protocol to use. The protocol must be supported by JVM.
+
+ <br />The reference list of protocols can be found in the
"Additional JSSE Standard Names"
+ section of the Java security guide. For Java 8, the list can be
found at
+ <a
href="https://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html#jssenames";>this</a>
+ page.
+ </td>
+ </tr>
+ <tr>
+ <td><code>${ns}.needClientAuth</code></td>
+ <td>false</td>
+ <td>Whether to require client authentication.</td>
+ </tr>
+ <tr>
+ <td><code>${ns}.trustStore</code></td>
+ <td>None</td>
+ <td>
+ Path to the trust store file. The path can be absolute or relative
to the directory in which
+ the process is started.
+ </td>
+ </tr>
+ <tr>
+ <td><code>${ns}.trustStorePassword</code></td>
+ <td>None</td>
+ <td>Password for the trust store.</td>
+ </tr>
+ <tr>
+ <td><code>${ns}.trustStoreType</code></td>
+ <td>JKS</td>
+ <td>The type of the trust store.</td>
+ </tr>
+</table>
+
+## Preparing the key stores
+
+Key stores can be generated by `keytool` program. The reference
documentation for this tool for
+Java 8 is
[here](https://docs.oracle.com/javase/8/docs/technotes/tools/unix/keytool.html).
+The most basic steps to configure the key stores and the trust store for a
Spark Standalone
+deployment mode is as follows:
+
+* Generate a key pair for each node
+* Export the public key of the key pair to a file on each node
+* Import all exported public keys into a single trust store
+* Distribute the trust store to the cluster nodes
### YARN mode
-The key-store can be prepared on the client side and then distributed and
used by the executors as the part of the application. It is possible because
the user is able to deploy files before the application is started in YARN by
using `spark.yarn.dist.files` or `spark.yarn.dist.archives` configuration
settings. The responsibility for encryption of transferring these files is on
YARN side and has nothing to do with Spark.
-For long-running apps like Spark Streaming apps to be able to write to
HDFS, it is possible to pass a principal and keytab to `spark-submit` via the
`--principal` and `--keytab` parameters respectively. The keytab passed in will
be copied over to the machine running the Application Master via the Hadoop
Distributed Cache (securely - if YARN is configured with SSL and HDFS
encryption is enabled). The Kerberos login will be periodically renewed using
this principal and keytab and the delegation tokens required for HDFS will be
generated periodically so the application can continue writing to HDFS.
+To provide a local trust store or key store file to drivers running in
cluster mode, they can be
+distributed with the application using the `--files` command line argument
(or the equivalent
+`spark.files` configuration). The files will be placed on the driver's
working directory, so the TLS
+configuration should just reference the file name with no absolute path.
+
+When distributing local key stores this way, make sure to configure the
underlying distributed
--- End diff --
may also add warning that these are distributed via underlying filesystem
(--files) so make sure its trusted/secure.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
