Github user ArtRand commented on a diff in the pull request:
https://github.com/apache/spark/pull/19272#discussion_r146472162
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala
---
@@ -213,6 +216,24 @@ private[spark] class
MesosCoarseGrainedSchedulerBackend(
sc.conf.getOption("spark.mesos.driver.frameworkId").map(_ + suffix)
)
+ // check that the credentials are defined, even though it's likely
that auth would have failed
+ // already if you've made it this far
+ if (principal != null && currentHadoopDelegationTokens.isDefined) {
+ logDebug(s"Principal found ($principal) starting token renewer")
+ // The renewal time is ignored when getting the initial delegation
tokens
+ // (CoarseGrainedSchedulerBackend.scala:getHadoopDelegationCreds),
so we get the renewal
+ // time here and schedule a thread to renew them.
+ val renewalTime =
--- End diff --
Ok, I agree. I refrained from touching the current code only because it was
added it so recently. I assumed the omission of renewal time was intentional
(or at least semi-intentional). I'll submit a refactor. Are you suggesting that
we make the renewal thread part of `CoarseGrainedSchedulerBackend` (with the
appropriate override for YARN/Mesos)? Or make a resource-manager specific
method?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
