This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new 286e3a8fb3c6 [SPARK-58152][K8S] Avoid logging credentials in
Kubernetes client config debug log
286e3a8fb3c6 is described below
commit 286e3a8fb3c67a09800372c8435363ca7b213cfe
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Jul 15 12:52:03 2026 -0700
[SPARK-58152][K8S] Avoid logging credentials in Kubernetes client config
debug log
Log only non-sensitive fields (`masterUrl`, `namespace`, `apiVersion`,
`requestTimeout`,
`connectionTimeout`, `trustCerts`) in the DEBUG log line of
`SparkKubernetesClientFactory.createKubernetesClient`, instead of
serializing the entire fabric8
`Config` object with a plain `ObjectMapper`.
`Config` holds the API server credentials, and a plain `ObjectMapper`
applies no masking, so the
previous line logged `oauthToken`, `clientKeyData`, `caCertData`,
`clientCertData`, the basic-auth
password, and the truststore password in cleartext. (`oauthToken` is
annotated `JsonIgnore`, but
`JsonIgnoreProperties(allowGetters = true)` on the class re-enables it via
its getter.)
`createKubernetesClient` runs on both the submission client and the driver,
and driver logs are
commonly collected and archived.
No. DEBUG log content only.
Pass the CIs.
Generated-by: Claude Fable 5
Closes #57281 from dongjoon-hyun/SPARK-58152.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit a4dc5d9499fa8d40b02a42d5d54dfbc1a1986f37)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git
a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala
b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala
index 557bf01cbdba..e3e84861d8e5 100644
---
a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala
+++
b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala
@@ -18,7 +18,6 @@ package org.apache.spark.deploy.k8s
import java.io.File
-import com.fasterxml.jackson.databind.ObjectMapper
import com.google.common.base.Charsets
import com.google.common.io.Files
import io.fabric8.kubernetes.client.{ConfigBuilder, KubernetesClient,
KubernetesClientBuilder}
@@ -109,8 +108,12 @@ object SparkKubernetesClientFactory extends Logging {
}.withOption(namespace) {
(ns, configBuilder) => configBuilder.withNamespace(ns)
}.build()
- logDebug("Kubernetes client config: " +
- new
ObjectMapper().writerWithDefaultPrettyPrinter().writeValueAsString(config))
+ // Log only non-sensitive fields. Serializing the whole Config would leak
credentials
+ // such as the OAuth token, client key, and certificate data.
+ logDebug(s"Kubernetes client config: masterUrl=${config.getMasterUrl}, " +
+ s"namespace=${config.getNamespace}, apiVersion=${config.getApiVersion},
" +
+ s"requestTimeout=${config.getRequestTimeout}, " +
+ s"connectionTimeout=${config.getConnectionTimeout},
trustCerts=${config.isTrustCerts}")
new KubernetesClientBuilder().withConfig(config).build()
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
