Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/14065#discussion_r71354857
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/security/AMDelegationTokenRenewer.scala
---
@@ -193,10 +196,17 @@ private[yarn] class AMDelegationTokenRenewer(
}
}
val nextSuffix = lastCredentialsFileSuffix + 1
+ // Time of next update should be later than renewal,
+ val timeOfNextUpdate =
+ (timeOfNextRenewal - System.currentTimeMillis()) * 1.1 +
System.currentTimeMillis()
--- End diff --
I'm a little concerned with the 1.1 here. If renewal is every 24 hours,
1.1 times that is another 144 minutes past renewal, this is fine for hdfs which
return 75% of renewal but its still quite long, it was 70 minutes before with
75% and 80% but I'm a little concerned with possibly other services and what
the user could return. So following my comment above perhaps we should move
the logic from hdfs credentials into here and do like the 75%/80% (or 85%)
again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
