This is an automated email from the ASF dual-hosted git repository.

LuciferYang pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new ebe55f7f9267 [SPARK-58872][K8S] Warn when driver credentials drop the 
driver service account
ebe55f7f9267 is described below

commit ebe55f7f9267e25ee038d6592bcb5dab5a52b4c4
Author: YangJie <[email protected]>
AuthorDate: Sat Aug 22 17:21:04 2026 +0800

    [SPARK-58872][K8S] Warn when driver credentials drop the driver service 
account
    
    ### What changes were proposed in this pull request?
    
    Log a warning when a submission sets both 
`spark.kubernetes.authenticate.driver.serviceAccountName` and any of the 
driver's own client credentials, since the credentials win and the service 
account is never applied to the driver pod. The message names the account that 
was dropped, the credential configs that took precedence, and the way to have 
both. It stays quiet when the pod spec already names that same account, because 
then nothing was lost. Also spell out the same in `running-on- [...]
    
    Behavior is unchanged. Two supporting bits: collecting the credential 
config names for the message replaces the four-way `isDefined` disjunction that 
computed `shouldMountSecret`, and `CONFIGS` joins `LogKeys` so the joined list 
gets a plural MDC key like `PATHS` and `EXECUTOR_IDS` rather than reusing 
`CONFIG2`, which everywhere else in the tree holds a single key.
    
    ### Why are the changes needed?
    
    `DriverKubernetesCredentialsFeatureStep.configurePod` applies the service 
account only on the branch with no credentials secret to mount:
    
    ```scala
    override def configurePod(pod: SparkPod): SparkPod = {
      if (!shouldMountSecret) {
        pod.copy(pod = buildPodWithServiceAccount(driverServiceAccount, 
pod).getOrElse(pod.pod))
      } else {
        // mounts the credentials secret; driverServiceAccount is never read 
here
    ```
    
    `shouldMountSecret` is true if any of `oauthToken`, `caCertFile`, 
`clientKeyFile`, `clientCertFile` was given for the driver.
    
    That precedence is intended. `docs/running-on-kubernetes.md` has said since 
2.3.0 that the service account "cannot be specified alongside a CA cert file, 
client key file, client cert file, and/or OAuth token", and the config's own 
doc says the driver "will favor using those credentials instead". But nothing 
enforces it and nothing reports it, so a submission setting both is accepted 
and looks honored, while the driver pod is left with the service account its 
spec already names, or the [...]
    
    `caCertFile` is the easiest of the four to trip over. It only establishes 
TLS trust in the API server, so it is not by itself something the driver 
authenticates with, yet it is enough on its own to drop the account:
    
    ```
    --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
    --conf spark.kubernetes.authenticate.driver.caCertFile=/path/to/ca.pem
    ```
    
    The driver then reaches the API server as the namespace default account, 
whose in-pod token `Config.autoConfigure` picks up.
    
    The warning points at `spark.kubernetes.authenticate.driver.mounted.*` 
rather than telling the user to drop one of the two. Those configs read paths 
already present in the driver pod, never feed `shouldMountSecret`, and 
therefore keep the service account, so they are the one setting that satisfies 
both intents. "Unset one of them" would have been bad advice in both 
directions: unsetting `caCertFile` gives up TLS trust in a private-CA API 
server, and unsetting the account also moves ex [...]
    
    One case is worth suppressing. A driver pod template that already names the 
same account in its spec loses nothing on this branch, since the branch keeps 
whatever the spec names, so warning there is only noise. It is also exactly the 
configuration a user lands on after reading the `mounted.*` advice and deciding 
to pin the account in a template instead. Reading the account off the spec 
needs both fields. `serviceAccount` is Kubernetes' deprecated alias of 
`serviceAccountName`, and `se [...]
    
    I went with a warning rather than a `require`. A `require` would match the 
documented "cannot" and is the more honest reading of the docs, but it would 
reject submissions that are accepted today, which needs a release note and a 
deprecation path. Happy to switch if reviewers would rather enforce it.
    
    ### Does this PR introduce _any_ user-facing change?
    
    A new WARN at submit time for submissions that set both. No API, 
configuration, or pod-spec change.
    
    ### How was this patch tested?
    
    Two tests in `DriverKubernetesCredentialsFeatureStepSuite`, split along 
what they pin.
    
    `warn when driver credentials drop the driver service account` sets the 
account plus a `caCertFile` and checks that the pod spec's `serviceAccount` and 
`serviceAccountName` are still null, pinning the behavior the warning 
describes; that exactly one WARN names both configs; that it does *not* name 
the three credential configs which were not set, which pins the filter rather 
than just the fact that something was logged; and that it still points at the 
`mounted.*` prefix, since that sen [...]
    
    `stay quiet when the driver service account survives` covers the three 
cases that must not warn: the account alone, where it *is* applied; the account 
alongside `mounted.caCertFile`, the escape hatch the message and the docs point 
at; and four pod template shapes naming the same account, covering 
`serviceAccountName` only, `serviceAccount` only, both, and the alias plus an 
empty `serviceAccountName`.
    
    Each assertion has a mutation that turns it red, and I ran all of them: 
dropping the new guard; dropping the `mounted.*` sentence; letting the 
`mounted.*` keys feed `shouldMountSecret`; reading only `getServiceAccount`; 
reading only `getServiceAccountName`; widening the guard to "the pod names any 
account"; dropping the empty-string filter; and reversing the two fields. One 
in the other direction as well: adding a `logInfo` to the step keeps all three 
quiet assertions green, since the [...]
    
    Against `master` the first test fails:
    
    ```
    [info] - SPARK-58872: warn when driver credentials drop the driver service 
account *** FAILED ***
    [info]   0 did not equal 1 expected one warning naming both 
spark.kubernetes.authenticate.driver.serviceAccountName and 
spark.kubernetes.authenticate.driver.caCertFile, got: List()
    ```
    
    `build/sbt -Pkubernetes kubernetes/test`: 386 tests across 42 suites, all 
passing. `common-utils/testOnly org.apache.spark.util.LogKeysSuite` passes, so 
`CONFIGS` is in sorted position. `kubernetes/scalastyle`, 
`kubernetes/Test/scalastyle`, and `common-utils-java/Compile/checkstyle` report 
no issues.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code (Opus 5)
    
    Closes #58163 from LuciferYang/fix-k8s-driver-sa-ignored.
    
    Authored-by: YangJie <[email protected]>
    Signed-off-by: yangjie01 <[email protected]>
    (cherry picked from commit 50c7626aa387454e3bb2c2074fc5841e5d4d91b2)
    Signed-off-by: yangjie01 <[email protected]>
---
 .../java/org/apache/spark/internal/LogKeys.java    |   2 +
 docs/running-on-kubernetes.md                      |  15 ++-
 .../DriverKubernetesCredentialsFeatureStep.scala   |  40 +++++-
 ...iverKubernetesCredentialsFeatureStepSuite.scala | 137 ++++++++++++++++++++-
 4 files changed, 184 insertions(+), 10 deletions(-)

diff --git 
a/common/utils-java/src/main/java/org/apache/spark/internal/LogKeys.java 
b/common/utils-java/src/main/java/org/apache/spark/internal/LogKeys.java
index bf53e4d2b773..b47bab42d6fa 100644
--- a/common/utils-java/src/main/java/org/apache/spark/internal/LogKeys.java
+++ b/common/utils-java/src/main/java/org/apache/spark/internal/LogKeys.java
@@ -127,6 +127,7 @@ public enum LogKeys implements LogKey {
   CONFIG3,
   CONFIG4,
   CONFIG5,
+  CONFIGS,
   CONFIG_DEPRECATION_MESSAGE,
   CONFIG_KEY_UPDATED,
   CONFIG_VERSION,
@@ -720,6 +721,7 @@ public enum LogKeys implements LogKey {
   SCHEMA,
   SCHEMA2,
   SERVER_NAME,
+  SERVICE_ACCOUNT_NAME,
   SERVICE_NAME,
   SERVLET_CONTEXT_HANDLER_PATH,
   SESSION_HANDLE,
diff --git a/docs/running-on-kubernetes.md b/docs/running-on-kubernetes.md
index 9679bc0c5f76..61d9fd04afa4 100644
--- a/docs/running-on-kubernetes.md
+++ b/docs/running-on-kubernetes.md
@@ -905,8 +905,13 @@ See the [configuration page](configuration.html) for 
information on Spark config
   <td><code>default</code></td>
   <td>
     Service account that is used when running the driver pod. The driver pod 
uses this service account when requesting
-    executor pods from the API server. Note that this cannot be specified 
alongside a CA cert file, client key file,
-    client cert file, and/or OAuth token. In client mode, use 
<code>spark.kubernetes.authenticate.serviceAccountName</code> instead.
+    executor pods from the API server. Note that this cannot be specified 
alongside a submitted CA cert file, client key
+    file, client cert file, and/or OAuth token: Spark mounts those as a secret 
and they take precedence, so the driver
+    pod is left with the service account its spec already names, or the 
namespace's default. Spark logs a warning when
+    the account is dropped. To have Spark apply this configuration anyway, put 
the
+    credentials inside the driver pod and point the
+    <code>spark.kubernetes.authenticate.driver.mounted.*</code> configurations 
at them instead, which does not mount a
+    secret. In client mode, use 
<code>spark.kubernetes.authenticate.serviceAccountName</code> instead.
   </td>
   <td>2.3.0</td>
 </tr>
@@ -1973,7 +1978,8 @@ See the below table for the full list of pod 
specifications that will be overwri
   <td>Value of 
<code>spark.kubernetes.authenticate.driver.serviceAccountName</code></td>
   <td>
     Spark will override <code>serviceAccount</code> with the value of the 
spark configuration for only
-    driver pods, and only if the spark configuration is specified. Executor 
pods will remain unaffected.
+    driver pods, and only if the spark configuration is specified and no 
driver credentials are
+    submitted for Spark to mount as a secret. Executor pods will remain 
unaffected.
   </td>
 </tr>
 <tr>
@@ -1981,7 +1987,8 @@ See the below table for the full list of pod 
specifications that will be overwri
   <td>Value of 
<code>spark.kubernetes.authenticate.driver.serviceAccountName</code></td>
   <td>
     Spark will override <code>serviceAccountName</code> with the value of the 
spark configuration for only
-    driver pods, and only if the spark configuration is specified. Executor 
pods will remain unaffected.
+    driver pods, and only if the spark configuration is specified and no 
driver credentials are
+    submitted for Spark to mount as a secret. Executor pods will remain 
unaffected.
   </td>
 </tr>
 <tr>
diff --git 
a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala
 
b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala
index 39dfec95176c..00680c3e319c 100644
--- 
a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala
+++ 
b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala
@@ -29,9 +29,11 @@ import org.apache.spark.deploy.k8s.{KubernetesConf, SparkPod}
 import org.apache.spark.deploy.k8s.Config._
 import org.apache.spark.deploy.k8s.Constants._
 import org.apache.spark.deploy.k8s.KubernetesUtils.buildPodWithServiceAccount
+import org.apache.spark.internal.Logging
+import org.apache.spark.internal.LogKeys.{CONFIG, CONFIGS, PREFIX, 
SERVICE_ACCOUNT_NAME, VALUE}
 
 private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: 
KubernetesConf)
-  extends KubernetesFeatureConfigStep {
+  extends KubernetesFeatureConfigStep with Logging {
 
   private val maybeMountedOAuthTokenFile = kubernetesConf.getOption(
     
s"$KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX.$OAUTH_TOKEN_FILE_CONF_SUFFIX")
@@ -59,10 +61,16 @@ private[spark] class 
DriverKubernetesCredentialsFeatureStep(kubernetesConf: Kube
     s"$KUBERNETES_AUTH_DRIVER_CONF_PREFIX.$CLIENT_CERT_FILE_CONF_SUFFIX",
     "Driver client cert file")
 
-  private val shouldMountSecret = oauthTokenBase64.isDefined ||
-    caCertDataBase64.isDefined ||
-    clientKeyDataBase64.isDefined ||
-    clientCertDataBase64.isDefined
+  private val submittedCredentialConfs = Seq(
+    OAUTH_TOKEN_CONF_SUFFIX -> oauthTokenBase64,
+    CA_CERT_FILE_CONF_SUFFIX -> caCertDataBase64,
+    CLIENT_KEY_FILE_CONF_SUFFIX -> clientKeyDataBase64,
+    CLIENT_CERT_FILE_CONF_SUFFIX -> clientCertDataBase64)
+    .collect { case (suffix, credential) if credential.isDefined =>
+      s"$KUBERNETES_AUTH_DRIVER_CONF_PREFIX.$suffix"
+    }
+
+  private val shouldMountSecret = submittedCredentialConfs.nonEmpty
 
   private val driverCredentialsSecretName =
     s"${kubernetesConf.resourceNamePrefix}-kubernetes-credentials"
@@ -71,6 +79,28 @@ private[spark] class 
DriverKubernetesCredentialsFeatureStep(kubernetesConf: Kube
     if (!shouldMountSecret) {
       pod.copy(pod = buildPodWithServiceAccount(driverServiceAccount, 
pod).getOrElse(pod.pod))
     } else {
+      // The credentials secret takes precedence over the driver service 
account: this branch never
+      // applies the account, so warn that the pod keeps whatever its spec 
names, or the namespace
+      // default. Stay quiet when the spec already names the same account. 
Both spec fields are
+      // read, `serviceAccountName` winning, matching Kubernetes' 
SetDefaults_PodSpec: a pod
+      // template is deserialized with no API-server defaulting, so it can 
leave either one null.
+      val podSpec = Option(pod.pod.getSpec)
+      val podServiceAccount = podSpec.flatMap(s => 
Option(s.getServiceAccountName))
+        .filter(_.nonEmpty)
+        .orElse(podSpec.flatMap(s => 
Option(s.getServiceAccount)).filter(_.nonEmpty))
+      driverServiceAccount.filterNot(podServiceAccount.contains).foreach { 
account =>
+        val keptAccount = podServiceAccount
+          .map(name => log"the pod keeps ${MDC(SERVICE_ACCOUNT_NAME, name)}, 
named by its spec")
+          .getOrElse(log"the pod falls back to the namespace's default 
account")
+        logWarning(log"Not applying " +
+          log"${MDC(CONFIG, 
KUBERNETES_DRIVER_SERVICE_ACCOUNT_NAME.key)}=${MDC(VALUE, account)} " +
+          log"to the driver pod, because the driver credentials given by " +
+          log"${MDC(CONFIGS, submittedCredentialConfs.mkString(", "))} take 
precedence: " +
+          keptAccount + log". To have Spark apply that configuration anyway, 
put the credentials " +
+          log"inside the driver pod and point the " +
+          log"${MDC(PREFIX, KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX)}.* 
configurations at " +
+          log"them instead, which does not mount a secret.")
+      }
       val driverPodWithMountedKubernetesCredentials =
         new PodBuilder(pod.pod)
           .editOrNewSpec()
diff --git 
a/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStepSuite.scala
 
b/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStepSuite.scala
index 33982e929888..eff0f2fb77a7 100644
--- 
a/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStepSuite.scala
+++ 
b/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStepSuite.scala
@@ -23,7 +23,8 @@ import java.util.Base64
 
 import scala.jdk.CollectionConverters._
 
-import io.fabric8.kubernetes.api.model.Secret
+import io.fabric8.kubernetes.api.model.{PodBuilder, Secret}
+import org.apache.logging.log4j.Level
 
 import org.apache.spark.{SparkConf, SparkFunSuite}
 import org.apache.spark.deploy.k8s.{KubernetesTestConf, SparkPod}
@@ -35,6 +36,8 @@ class DriverKubernetesCredentialsFeatureStepSuite extends 
SparkFunSuite {
 
   private val credentialsTempDirectory = Utils.createTempDir()
   private val BASE_DRIVER_POD = SparkPod.initialPod()
+  private val SERVICE_ACCOUNT_CONF = KUBERNETES_DRIVER_SERVICE_ACCOUNT_NAME.key
+  private val STEP_LOGGER = 
classOf[DriverKubernetesCredentialsFeatureStep].getName
 
   test("Don't set any credentials") {
     val kubernetesConf = KubernetesTestConf.createDriverConf()
@@ -127,6 +130,138 @@ class DriverKubernetesCredentialsFeatureStepSuite extends 
SparkFunSuite {
     assert(driverContainerVolumeMount.head.getMountPath === 
DRIVER_CREDENTIALS_SECRETS_BASE_DIR)
   }
 
+  test("SPARK-58872: warn when driver credentials drop the driver service 
account") {
+    val caCertConf = 
s"$KUBERNETES_AUTH_DRIVER_CONF_PREFIX.$CA_CERT_FILE_CONF_SUFFIX"
+    val caCertFile = writeCredentials("sa-ca.pem", "ca-cert")
+    val stepUnderTest = stepWith(
+      SERVICE_ACCOUNT_CONF -> "spark", caCertConf -> 
caCertFile.getAbsolutePath)
+    val logAppender = new LogAppender
+    val configuredPod = withLogAppenderReturning(logAppender) {
+      stepUnderTest.configurePod(BASE_DRIVER_POD)
+    }
+    // The documented behavior the warning describes: the credentials win, so 
the account is
+    // never applied, and with no account on the spec the pod falls back to 
the namespace default.
+    assert(configuredPod.pod.getSpec.getServiceAccount === null)
+    assert(configuredPod.pod.getSpec.getServiceAccountName === null)
+    val warnings = warningsFrom(logAppender)
+    val named = warnings.filter(w => w.contains(SERVICE_ACCOUNT_CONF) && 
w.contains(caCertConf))
+    assert(named.size === 1, s"expected one warning naming both 
$SERVICE_ACCOUNT_CONF and " +
+      s"$caCertConf, got: $warnings")
+    assert(named.head.contains("namespace's default"),
+      s"warning does not say what the pod falls back to: ${named.head}")
+    // Only the credentials actually submitted are named, so a message that 
lists all four fails.
+    Seq(OAUTH_TOKEN_CONF_SUFFIX, CLIENT_KEY_FILE_CONF_SUFFIX, 
CLIENT_CERT_FILE_CONF_SUFFIX)
+      .map(suffix => s"$KUBERNETES_AUTH_DRIVER_CONF_PREFIX.$suffix")
+      .foreach(conf => assert(!named.head.contains(conf),
+        s"warning names $conf, which was not set: ${named.head}"))
+    // The way out of the conflict is worth spelling out, so require it stays 
in the message.
+    
assert(named.head.contains(s"$KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX.*"),
+      s"warning does not point at the mounted configs: ${named.head}")
+
+    // A template naming a different account does lose the configured one, so 
it must warn and say
+    // which account the pod keeps. The second case pins `serviceAccountName` 
beating the alias.
+    Seq(
+      podWithAccount(serviceAccountName = Some("other")),
+      podWithAccount(serviceAccount = Some("spark"), serviceAccountName = 
Some("other"))
+    ).foreach { otherAccountPod =>
+      val otherAppender = new LogAppender
+      withLogAppender(otherAppender, loggerNames = Seq(STEP_LOGGER)) {
+        stepUnderTest.configurePod(otherAccountPod)
+      }
+      val otherWarnings = 
warningsFrom(otherAppender).filter(_.contains(SERVICE_ACCOUNT_CONF))
+      assert(otherWarnings.size === 1,
+        s"expected one warning for a pod running as " +
+          s"${otherAccountPod.pod.getSpec.getServiceAccountName}/" +
+          s"${otherAccountPod.pod.getSpec.getServiceAccount}, got: 
$otherWarnings")
+      assert(otherWarnings.head.contains("other"),
+        s"warning does not name the account the pod keeps: 
${otherWarnings.head}")
+    }
+  }
+
+  test("SPARK-58872: stay quiet when the driver service account survives") {
+    // With the account alone there is nothing to mount, so it is applied and 
nothing is warned.
+    val saOnlyAppender = new LogAppender
+    val saOnlyPod = withLogAppenderReturning(saOnlyAppender) {
+      stepWith(SERVICE_ACCOUNT_CONF -> "spark").configurePod(BASE_DRIVER_POD)
+    }
+    assert(saOnlyPod.pod.getSpec.getServiceAccount === "spark")
+    assert(saOnlyPod.pod.getSpec.getServiceAccountName === "spark")
+    assert(warningsFrom(saOnlyAppender).isEmpty,
+      s"nothing was dropped, so nothing to warn about: 
${warningsFrom(saOnlyAppender)}")
+
+    // The mounted configs are the escape hatch the message and the docs point 
at, so setting one
+    // alongside the account must neither drop it nor warn. The mounted keys 
never feed
+    // `shouldMountSecret`, so this takes the first branch and never reaches 
the guard.
+    val mountedAppender = new LogAppender
+    val mountedPod = withLogAppenderReturning(mountedAppender) {
+      stepWith(SERVICE_ACCOUNT_CONF -> "spark",
+        
s"$KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX.$CA_CERT_FILE_CONF_SUFFIX" -> 
"/etc/ca.pem")
+        .configurePod(BASE_DRIVER_POD)
+    }
+    assert(mountedPod.pod.getSpec.getServiceAccount === "spark")
+    assert(mountedPod.pod.getSpec.getServiceAccountName === "spark")
+    assert(warningsFrom(mountedAppender).isEmpty,
+      s"mounted configs must not drop the account: 
${warningsFrom(mountedAppender)}")
+
+    // A pod template that already names the same account loses nothing, so 
there is nothing to say.
+    // An explicitly empty `serviceAccountName` means unset, so Kubernetes 
copies the alias up.
+    val caCertFile = writeCredentials("quiet-ca.pem", "ca-cert")
+    val stepUnderTest = stepWith(SERVICE_ACCOUNT_CONF -> "spark",
+      s"$KUBERNETES_AUTH_DRIVER_CONF_PREFIX.$CA_CERT_FILE_CONF_SUFFIX" ->
+        caCertFile.getAbsolutePath)
+    Seq(
+      podWithAccount(serviceAccountName = Some("spark")),
+      podWithAccount(serviceAccount = Some("spark")),
+      podWithAccount(serviceAccount = Some("spark"), serviceAccountName = 
Some("spark")),
+      podWithAccount(serviceAccount = Some("spark"), serviceAccountName = 
Some(""))
+    ).foreach { templatePod =>
+      val sameAccountAppender = new LogAppender
+      withLogAppender(sameAccountAppender, loggerNames = Seq(STEP_LOGGER)) {
+        stepUnderTest.configurePod(templatePod)
+      }
+      assert(warningsFrom(sameAccountAppender).isEmpty,
+        s"warned although the pod already runs as spark via " +
+          s"${templatePod.pod.getSpec.getServiceAccountName}/" +
+          s"${templatePod.pod.getSpec.getServiceAccount}: " +
+          s"${warningsFrom(sameAccountAppender)}")
+    }
+  }
+
+  private def stepWith(confs: (String, String)*): 
DriverKubernetesCredentialsFeatureStep = {
+    val sparkConf = new SparkConf(false)
+    confs.foreach { case (k, v) => sparkConf.set(k, v) }
+    new DriverKubernetesCredentialsFeatureStep(
+      KubernetesTestConf.createDriverConf(sparkConf = sparkConf))
+  }
+
+  /**
+   * A driver pod whose spec names a service account, standing in for a user 
pod template. A
+   * template is parsed with no API-server defaulting, so it can name either 
field on its own.
+   */
+  private def podWithAccount(
+      serviceAccount: Option[String] = None,
+      serviceAccountName: Option[String] = None): SparkPod = {
+    val spec = new PodBuilder(BASE_DRIVER_POD.pod).editOrNewSpec()
+    serviceAccount.foreach(spec.withServiceAccount(_))
+    serviceAccountName.foreach(spec.withServiceAccountName(_))
+    SparkPod(spec.endSpec().build(), BASE_DRIVER_POD.container)
+  }
+
+  private def warningsFrom(appender: LogAppender): Seq[String] =
+    appender.loggingEvents
+      .filter(_.getLevel === Level.WARN)
+      .map(_.getMessage.getFormattedMessage)
+      .toSeq
+
+  /** `withLogAppender` returns Unit, so carry the block's value out of it. */
+  private def withLogAppenderReturning[T](appender: LogAppender)(f: => T): T = 
{
+    var result: Option[T] = None
+    withLogAppender(appender, loggerNames = Seq(STEP_LOGGER)) {
+      result = Some(f)
+    }
+    result.get
+  }
+
   private def writeCredentials(credentialsFileName: String, 
credentialsContents: String): File = {
     val credentialsFile = new File(credentialsTempDirectory, 
credentialsFileName)
     Files.writeString(credentialsFile.toPath, credentialsContents)


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to