Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4688#discussion_r28728492
--- Diff: docs/security.md ---
@@ -31,6 +31,7 @@ SSL must be configured on each node and configured for
each component involved i
### YARN mode
The key-store can be prepared on the client side and then distributed and
used by the executors as the part of the application. It is possible because
the user is able to deploy files before the application is started in YARN by
using `spark.yarn.dist.files` or `spark.yarn.dist.archives` configuration
settings. The responsibility for encryption of transferring these files is on
YARN side and has nothing to do with Spark.
+For long-running apps like Spark Streaming apps be able to write to HDFS,
it is possible to pass a principal and keytab to `spark-submit` via the
`--principal` and `--keytab` parameters respectively. The keytab passed in will
be copied over to the machine running the Application Master securely via the
Hadoop Distributed Cache. The Kerberos login will be periodically renewed using
this principal and keytab and the delegation tokens required for HDFS will be
generated periodically so the application can continue writing to HDFS.
--- End diff --
"...apps like Spark Streaming to be able..."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
