Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19272#discussion_r148342319
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -102,8 +99,15 @@ class CoarseGrainedSchedulerBackend(scheduler:
TaskSchedulerImpl, val rpcEnv: Rp
// hadoop token manager used by some sub-classes (e.g. Mesos)
def hadoopDelegationTokenManager: Option[HadoopDelegationTokenManager] =
None
- // Hadoop delegation tokens to be sent to the executors.
- val hadoopDelegationCreds: Option[Array[Byte]] =
getHadoopDelegationCreds()
+ def hadoopCredentialRenewer: Option[HadoopCredentialRenewer] = None
+
+ // Hadoop delegation tokens to be sent to the executors, can be updated
as necessary.
+ var renewableDelegationTokens: Option[RenewableDelegationTokens] =
--- End diff --
This is another part where the interface is brittle and a bit confusing.
This should really be a private field, because there should be a single way for
sub-classes to update it (send a `UpdateDelegationTokens` message). But it's
not because Mesos needs to poke at it because the rest of the API is all a bit
weird.
If instead you follow my original suggestion (have a abstract
`initializeDelegationTokens()` method that Mesos overrides), then everything
becomes a lot more localized and cleaner.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
