tgravescs commented on a change in pull request #31761:
URL: https://github.com/apache/spark/pull/31761#discussion_r594697177
##########
File path: docs/security.md
##########
@@ -838,6 +838,17 @@ The following options provides finer-grained control for
this feature:
</td>
<td>3.0.0</td>
</tr>
+<tr>
+ <td><code>spark.kerberos.renewal.exclude.hadoopFileSystems</code></td>
+ <td>(none)</td>
+ <td>
+ A comma-separated list of Hadoop filesystems for whose hosts will be
excluded from from delegation
+ token renewal at resource scheduler. For example,
<code>spark.kerberos.renewal.exclude.hadoopFileSystems=hdfs://nn1.com:8032,
+ hdfs://nn2.com:8032</code>. This is known to work under YARN for now, so
YARN Resource Manager won't renew tokens for the application.
+ Note that as resource scheduler does not renew token, the application
might not be long running once the token expires.
Review comment:
oh right. Please update the description on the issue to be what you are
really trying to do here.
I'm not sure that is completely true though either. getTokenRenewalInterval
fetches the delegation tokens in fetchDelegationTokens and then calls renew on
them to get the next expiration time. I'm assuming your changes in
fetchDelegationTokens makes renew not work. What does renew return in this case
that renewer is empty? It looks like it should throw an exception going by the
base Hadoop classes I looked at.
Assuming its not the actual next renewal time since you say it doesn't get
renewed if this tokens expiration is actually less than all the other tokens
than we won't properly renew it on the Spark side either. That is in the case
the key tab was specified. If it throws an exception that is even worse.
I'm also wondering out loud if this config would ever apply to anything
other than YARN. there is no renewal component in k8s that I'm aware of
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
