parthchandra commented on code in PR #57285:
URL: https://github.com/apache/spark/pull/57285#discussion_r3678382565
##########
core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala:
##########
@@ -293,14 +288,19 @@ private[spark] class HadoopDelegationTokenManager(
"credentials are available and direct credential providers are not
enabled.")
}
+ // If every direct provider failed and no tokens were obtained, throw so
that
+ // updateTokensTask() skips distributing empty credentials and schedules a
retry.
+ // This mirrors the Kerberos path, where a provider failure propagates
instead of
+ // sending empty tokens to executors.
+ if (failureCount > 0 && nextRenewal == Long.MaxValue) {
Review Comment:
You're right, and that's exactly the case @peter-toth flagged.
Applied your suggestion to key the check off
creds.numberOfTokens()/numberOfSecretKeys() so a partial success is never
discarded. Also added a `TestNoExpiryProvider` (adds a credential, returns
None) and a partial-success test asserting the fetched token is still
distributed when another provider fails, plus tightened the total-failure test
so it's actually empty
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
