This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new cd92a0c [SPARK-55094] Enable `spark.authenticate` by default
cd92a0c is described below
commit cd92a0ca60229b0bf773a5d342766da3a9a4ca39
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Jan 20 16:46:35 2026 +0900
[SPARK-55094] Enable `spark.authenticate` by default
### What changes were proposed in this pull request?
This PR aims to enable `spark.authenticate` by default.
### Why are the changes needed?
To improve Apache Spark Application security by default.
### Does this PR introduce _any_ user-facing change?
Spark will also automatically generate an authentication secret unique to
each application. The secret is propagated to executor pods using environment
variables. This means that any user that can list pods in the namespace where
the Spark application is running can also see their authentication secret.
Access control rules should be properly set up by the
Kubernetes admin to ensure that Spark authentication is secure.
### How was this patch tested?
Manual review.
**1. Run `Spark Connect Server` and check `authentication` log**
```
$ kubectl apply -f examples/spark-connect-server.yaml
$ kubectl logs spark-connect-server-0-driver | grep authentication | head
-n1
26/01/20 07:25:44 INFO SecurityManager: SecurityManager: authentication
enabled; ui acls disabled; users with view permissions: spark groups with view
permissions: EMPTY; users with modify permissions: spark; groups with modify
permissions: EMPTY; RPC SSL disabled
```
**2. Run `Swift Job` and check the result**
```
$ kubectl apply -f examples/job/pi-swift.yaml
job.batch/spark-connect-swift-pi created
$ kubectl logs -f spark-connect-swift-pi-7spx4
Pi is roughly 3.1433951433951433
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #466 from dongjoon-hyun/SPARK-55094.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java | 1 +
1 file changed, 1 insertion(+)
diff --git
a/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java
b/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java
index ddda781..91f4289 100644
---
a/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java
+++
b/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java
@@ -165,6 +165,7 @@ public class SparkAppSubmissionWorker {
sparkMasterUrlPrefix +
"https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT";);
String appId = generateSparkAppId(app);
effectiveSparkConf.setIfMissing("spark.app.id", appId);
+ effectiveSparkConf.setIfMissing("spark.authenticate", "true");
return SparkAppDriverConf.create(
effectiveSparkConf,
sparkVersion,
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
