tgravescs commented on a change in pull request #31761:
URL: https://github.com/apache/spark/pull/31761#discussion_r594351479
##########
File path: docs/security.md
##########
@@ -838,6 +838,17 @@ The following options provides finer-grained control for
this feature:
</td>
<td>3.0.0</td>
</tr>
+<tr>
+ <td><code>spark.kerberos.renewal.exclude.hadoopFileSystems</code></td>
+ <td>(none)</td>
+ <td>
+ A comma-separated list of Hadoop filesystems for whose hosts will be
excluded from from delegation
+ token renewal at resource scheduler. For example,
<code>spark.kerberos.renewal.exclude.hadoopFileSystems=hdfs://nn1.com:8032,
+ hdfs://nn2.com:8032</code>. This is known to work under YARN for now, so
YARN Resource Manager won't renew tokens for the application.
+ Note that as resource scheduler does not renew token, the application
might not be long running once the token expires.
Review comment:
so my intent was actually to add more description up on like line 792
and add to the "When using a Hadoop filesystem " bit.
Perhaps something like:
service hosting the user's home directory and staging directory. Spark will
renew Hadoop filesystem delegation tokens before their expiration unless the
token is excluded via spark.kerberos.renewal.exclude.hadoopFileSystems. Please
note that if the token is not renewed, any application that attempts to access
the filesystem associated with that token after it expired will likely fail.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
