viirya commented on a change in pull request #31761:
URL: https://github.com/apache/spark/pull/31761#discussion_r594749886
##########
File path: docs/security.md
##########
@@ -838,6 +838,17 @@ The following options provides finer-grained control for
this feature:
</td>
<td>3.0.0</td>
</tr>
+<tr>
+ <td><code>spark.kerberos.renewal.exclude.hadoopFileSystems</code></td>
+ <td>(none)</td>
+ <td>
+ A comma-separated list of Hadoop filesystems for whose hosts will be
excluded from from delegation
+ token renewal at resource scheduler. For example,
<code>spark.kerberos.renewal.exclude.hadoopFileSystems=hdfs://nn1.com:8032,
+ hdfs://nn2.com:8032</code>. This is known to work under YARN for now, so
YARN Resource Manager won't renew tokens for the application.
+ Note that as resource scheduler does not renew token, the application
might not be long running once the token expires.
Review comment:
Okay, this sounds a good point to me. I think I should not change the
behavior of `getTokenRenewalInterval` because it is not related to the issue
here. The renew call in `getTokenRenewalInterval` is just used for obtaining
renewal interval for all FS tokens.
I assume this config is only for YARN-specific behavior. I documented it
explicitly about YARN in the config doc/security.md. Is any other thing I
should do for it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
