Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22624#discussion_r223590938
--- Diff:
core/src/main/scala/org/apache/spark/deploy/security/AbstractCredentialRenewer.scala
---
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.deploy.security
+
+import java.io.File
+import java.security.PrivilegedExceptionAction
+import java.util.concurrent.{ScheduledExecutorService, TimeUnit}
+import java.util.concurrent.atomic.AtomicReference
+
+import org.apache.hadoop.conf.Configuration
+import org.apache.hadoop.security.{Credentials, UserGroupInformation}
+
+import org.apache.spark.SparkConf
+import org.apache.spark.deploy.SparkHadoopUtil
+import org.apache.spark.internal.Logging
+import org.apache.spark.internal.config._
+import org.apache.spark.rpc.RpcEndpointRef
+import
org.apache.spark.scheduler.cluster.CoarseGrainedClusterMessages.UpdateDelegationTokens
+import org.apache.spark.ui.UIUtils
+import org.apache.spark.util.ThreadUtils
+
+/**
+ * Base class for periodically updating delegation tokens needed by the
application.
+ *
+ * When configured with a principal and a keytab, this manager will make
sure long-running apps
+ * (such as Spark Streaming apps) can run without interruption while
accessing secured services. It
+ * periodically logs in to the KDC with user-provided credentials, and
contacts all the configured
+ * secure services to obtain delegation tokens to be distributed to the
rest of the application.
+ *
+ * This class will manage the kerberos login, by renewing the TGT when
needed. Because the UGI API
+ * does not expose the TTL of the TGT, a configuration controls how often
to check that a relogin is
+ * necessary. This is done reasonably often since the check is a no-op
when the relogin is not yet
+ * needed. The check period can be overridden in the configuration.
+ *
+ * New delegation tokens are created once 75% of the renewal interval of
the original tokens has
+ * elapsed. The new tokens are sent to the Spark driver endpoint once it's
registered with the AM.
+ * The driver is tasked with distributing the tokens to other processes
that might need them.
+ *
+ * This class can also be used when without a principal and keytab, in
which case token renewal will
+ * not be available. It provides a different API in that case (see
`createAndUpdateTokens()`), which
+ * automates the distribution of tokens to the different processes in the
Spark app.
+ */
+private[spark] abstract class AbstractCredentialRenewer(
+ protected val sparkConf: SparkConf,
+ protected val hadoopConf: Configuration) extends Logging {
+
+ private val principal = sparkConf.get(PRINCIPAL).orNull
+ private val keytab = sparkConf.get(KEYTAB).orNull
+
+ if (principal != null) {
+ require(keytab != null, "Kerberos principal specified without a
keytab.")
+ require(new File(keytab).isFile(), s"Cannot find keytab at $keytab.")
+ }
+
+ private val renewalExecutor: ScheduledExecutorService =
--- End diff --
Why not a typesafe Option here?
If `createAndUpdateTokens()` is extracted this would be always defined.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
