Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22915#discussion_r230140235
--- Diff: docs/security.md ---
@@ -798,6 +782,50 @@ achieved by setting
`spark.kubernetes.hadoop.configMapName` to a pre-existing Co
local:///opt/spark/examples/jars/spark-examples_<VERSION>.jar \
<HDFS_FILE_LOCATION>
```
+
+## Long-Running Applications
+
+Long-running applications may run into issues if their run time exceeds
the maximum delegation
+token lifetime configured in services it needs to access.
+
+Spark supports automatically creating new tokens for these applications
when running in YARN, Mesos, and Kubernetes modes.
+If one wishes to launch the renewal thread in the Driver, Kerberos
credentials need to be provided to the Spark application
+via the `spark-submit` command, using the `--principal` and `--keytab`
parameters.
+
+The provided keytab will be copied over to the machine running the
Application Master via the Hadoop
+Distributed Cache. For this reason, it's strongly recommended that both
YARN and HDFS be secured
+with encryption, at least.
+
+The Kerberos login will be periodically renewed using the provided
credentials, and new delegation
+tokens for supported will be created.
+
+#### Long-Running Kerberos in Kubernetes
+
+This section addresses the additional feature added uniquely to
Kubernetes. If you are running an external token service
--- End diff --
I don't want us to support internal proprietary services. If a company has
an internal proprietary service for this, they can also spend their own
resources to customize Spark so it does what they need.
We (the Spark community) cannot maintain compatibility and make sure code
keeps working with things we don't know exist. Which is why I've always
complained when you mention this service.
> it clearly seems necessary to define how such a service should function
as well. that would still be a bit hand-wavy.
It can't be hand wavy. Otherwise if someone tells us their own custom
implementation of that service doesn't work, whose fault it is?
Again, we just can't write code against something that does not exist.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
