Great to see the 1.0 release out! Thanks Dongjoon for driving the release, and thanks everyone who contributed along the way!
*Zhou JIANG* On Sun, Jul 26, 2026 at 15:47 Dongjoon Hyun <[email protected]> wrote: > Hi all, > > On behalf of the Apache Spark PMC, I am proud to announce the release > of Apache Spark K8s Operator 1.0.0! > > This is the first stable release of the Apache Spark K8s Operator, an > official subproject of Apache Spark. It marks a major milestone for the > Spark community: for the first time, the Apache Spark project itself > provides a complete, community-driven, vendor-neutral standard for > running Spark on Kubernetes via the Operator pattern. > > Built on the Java Operator SDK and the Fabric8 Kubernetes client, the > operator manages the full lifecycle of Spark workloads through two > Kubernetes custom resources: > > - SparkApplication: submit and manage individual Spark applications > (Java/Scala, PySpark, and SQL) with a rich state machine covering > submission, execution, failure handling, retries, and resource > cleanup. > - SparkCluster: provision and operate standalone Spark clusters, > including Spark Connect servers, Spark History Server, and > Spark Thrift Server deployments, with StatefulSet-based masters > and workers and Horizontal Pod Autoscaler support. > > Highlights of 1.0.0: > > - Supports Apache Spark 4.0 and above and > recent Kubernetes versions (1.34+ recommended). > - First-class Spark Connect support for next-generation > client-server Spark workloads. > - Helm-chart based installation, published on Artifact Hub, with > namespace-scoped RBAC and multi-namespace watching. > - Gateway API (HTTPRoute/GRPCRoute) integration for exposing > driver UIs and Spark Connect endpoints. > - Production-grade operations: metrics, health probes, leader > election, and extensive end-to-end test coverage via Chainsaw. > > Just as importantly, 1.0.0 was designed and verified as the hub of the > broader Apache Spark ecosystem. Out of the box, it ships with working > examples and e2e coverage for: > > - Apache Iceberg table format with Spark Connect > - Apache Hadoop-compatible storage and S3-compatible object stores > - Apache DataFusion Comet and Apache Gluten native accelerators > - Apache Celeborn remote shuffle service > - Apache YuniKorn and Volcano batch schedulers > > We believe this release establishes the standard way to run the entire > Spark ecosystem on Kubernetes, and we are proud of what the community > has built together. > > To get started: > > helm repo add spark https://apache.github.io/spark-kubernetes-operator > helm repo update > helm install spark spark/spark-kubernetes-operator > > Release Note: > > - https://github.com/apache/spark-kubernetes-operator/releases/tag/1.0.0 > - https://github.com/apache/spark-kubernetes-operator/milestone/3 > - https://s.apache.org/spark-kubernetes-operator-1.0.0 > > Published Docker Image: > > - apache/spark-kubernetes-operator:1.0.0 > > Useful links: > > - Download: > https://downloads.apache.org/spark/spark-kubernetes-operator-1.0.0/ > - Website: https://s.apache.org/spark-kubernetes-operator/ > - Source: https://github.com/apache/spark-kubernetes-operator > - Artifact Hub: > https://artifacthub.io/packages/search?repo=spark-kubernetes-operator > - Documentation: > https://github.com/apache/spark-kubernetes-operator/tree/main/docs > > We would like to thank all the contributors, reviewers, and users who > made this first release possible. This release is the result of the > Apache Spark community's collaborative effort, and we look forward to > your feedback and contributions. > > Regards, > Dongjoon Hyun > on behalf of the Apache Spark PMC >
