+1 On Thu, Nov 9, 2023 at 7:57 PM Chao Sun <sunc...@apache.org> wrote: > > +1 > > > On Thu, Nov 9, 2023 at 6:36 PM Xiao Li <gatorsm...@gmail.com> wrote: > > > > +1 > > > > huaxin gao <huaxin.ga...@gmail.com> 于2023年11月9日周四 16:53写道： > >> > >> +1 > >> > >> On Thu, Nov 9, 2023 at 3:14 PM DB Tsai <dbt...@dbtsai.com> wrote: > >>> > >>> +1 > >>> > >>> To be completely transparent, I am employed in the same department as > >>> Zhou at Apple. > >>> > >>> I support this proposal, provided that we witness community adoption > >>> following the release of the Flink Kubernetes operator, streamlining > >>> Flink deployment on Kubernetes. > >>> > >>> A well-maintained official Spark Kubernetes operator is essential for our > >>> Spark community as well. > >>> > >>> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1 > >>> > >>> On Nov 9, 2023, at 12:05 PM, Zhou Jiang <zhou.c.ji...@gmail.com> wrote: > >>> > >>> Hi Spark community, > >>> > >>> I'm reaching out to initiate a conversation about the possibility of > >>> developing a Java-based Kubernetes operator for Apache Spark. Following > >>> the operator pattern > >>> (https://kubernetes.io/docs/concepts/extend-kubernetes/operator/), Spark > >>> users may manage applications and related components seamlessly using > >>> native tools like kubectl. The primary goal is to simplify the Spark user > >>> experience on Kubernetes, minimizing the learning curve and operational > >>> complexities and therefore enable users to focus on the Spark application > >>> development. > >>> Although there are several open-source Spark on Kubernetes operators > >>> available, none of them are officially integrated into the Apache Spark > >>> project. As a result, these operators may lack active support and > >>> development for new features. Within this proposal, our aim is to > >>> introduce a Java-based Spark operator as an integral component of the > >>> Apache Spark project. This solution has been employed internally at Apple > >>> for multiple years, operating millions of executors in real production > >>> environments. The use of Java in this solution is intended to accommodate > >>> a wider user and contributor audience, especially those who are familiar > >>> with Scala. > >>> Ideally, this operator should have its dedicated repository, similar to > >>> Spark Connect Golang or Spark Docker, allowing it to maintain a loose > >>> connection with the Spark release cycle. This model is also followed by > >>> the Apache Flink Kubernetes operator. > >>> We believe that this project holds the potential to evolve into a > >>> thriving community project over the long run. A comparison can be drawn > >>> with the Flink Kubernetes Operator: Apple has open-sourced internal Flink > >>> Kubernetes operator, making it a part of the Apache Flink project > >>> (https://github.com/apache/flink-kubernetes-operator). This move has > >>> gained wide industry adoption and contributions from the community. In a > >>> mere year, the Flink operator has garnered more than 600 stars and has > >>> attracted contributions from over 80 contributors. This showcases the > >>> level of community interest and collaborative momentum that can be > >>> achieved in similar scenarios. > >>> More details can be found at SPIP doc : Spark Kubernetes Operator > >>> https://docs.google.com/document/d/1f5mm9VpSKeWC72Y9IiKN2jbBn32rHxjWKUfLRaGEcLE > >>> > >>> Thanks, > >>> > >>> -- > >>> Zhou JIANG > >>> > >>> > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >
--------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
