Github user harishreedharan commented on a diff in the pull request:
https://github.com/apache/spark/pull/9168#discussion_r42421617
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/AMDelegationTokenRenewer.scala
---
@@ -177,6 +177,7 @@ private[yarn] class AMDelegationTokenRenewer(
})
// Add the temp credentials back to the original ones.
UserGroupInformation.getCurrentUser.addCredentials(tempCreds)
+ SparkHadoopUtil.get.updateCurrentUserHDFSDelegationToken()
--- End diff --
So I don't understand this. We are already getting new tokens for what I
assume is the primary namenode using the `obtainTokensForNamenodes` method
call, correct? Is this only to get the tokens for the standby?
I don't think this will work though - You'd need to do the same thing we
do for the primary (add the tokens to a new UGI and copy the credentials from
there). From what I can remember, when you add new tokens for a service that
already has tokens - the tokens won't get overwritten, even if they are expired
- which is why we went with creating a new UGI and adding those credentials
over (which would overwrite the old tokens). Are you sure the
`HAUtil.cloneDelegationTokenForLogicalUri` will overwrite the tokens?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
