dongjoon-hyun opened a new pull request, #702: URL: https://github.com/apache/spark-kubernetes-operator/pull/702
### What changes were proposed in this pull request? This PR sets `spark.kubernetes.executor.useDriverPodIP` to `true` by default via `setIfMissing` in `SparkAppSubmissionWorker#buildDriverConf`, next to the other operator-injected defaults. Since it uses `setIfMissing`, an explicit value in the application spec is always respected. ### Why are the changes needed? `spark.kubernetes.executor.useDriverPodIP` (added in Spark 4.1.0 via SPARK-53944) makes executor pods connect to the driver using the driver pod IP instead of the driver `Service` hostname, bypassing Kubernetes DNS. The operator fully manages the driver pod lifecycle, and in cluster mode the driver is not restarted (the application terminates if it dies), so the pod IP is stable for the application's lifetime. Enabling it by default avoids DNS propagation delays/failures on executor startup and reduces CoreDNS load. It is a no-op for Spark versions older than 4.1.0. ### Does this PR introduce _any_ user-facing change? Yes. When `spark.kubernetes.executor.useDriverPodIP` is unset, the operator now defaults it to `true` (previously Spark's default of `false`). Applications that set it explicitly are unaffected. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
