huangzhir opened a new pull request, #36240: URL: https://github.com/apache/spark/pull/36240
### What changes were proposed in this pull request? This PR is add a judgment. If the renewable delegationTokenProvider returns None, changes the result renewal time to the current time + delay（sparkconf spark.security.credentials.retryWait）. ### Why are the changes needed? HadoopFSDelegationTokenProvider.obtaindelegationtokens returns None when an exception is encountered. HadoopDelegationTokenProvider that cannot be renewed also returns None. If the obtaindelegationtokens of all HadoopDelegationTokenProvider return None,HadoopDelegationTokenManager in obtaindelegationtokens, call foldleft (long. Maxvalue) (math. Min) to return long.MaxValue. Therefore, the log shows "scheduling renewal in 1921535501304.2 H" At this time, HadoopFSDelegationToken cannot be renewed in time, resulting in throw "hdfs_delete_token not found in cache" Exception ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? For testability, modify the method obtaindelegationtokens protected and modify the parameters. A method is added to the HadoopDelegationTokenManagerSuite to test Hadoop delegationtokenmanager.obtaindelegationtokens method. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
