Can you share link to the source? בתאריך יום ד׳, 23 בפבר׳ 2022, 6:52, מאת bo yang <bobyan...@gmail.com>:
> We do not have SaaS yet. Now it is an open source project we build in our > part time , and we welcome more people working together on that. > > You could specify cluster size (EC2 instance type and number of instances) > and run it for 1 hour. Then you could run one click command to destroy the > cluster. It is possible to merge these steps as well, and provide a > "serverless" experience. That is in our TODO list :) > > > On Tue, Feb 22, 2022 at 8:36 PM Bitfox <bit...@bitfox.top> wrote: > >> How can I specify the cluster memory and cores? >> For instance, I want to run a job with 16 cores and 300 GB memory for >> about 1 hour. Do you have the SaaS solution for this? I can pay as I did. >> >> Thanks >> >> On Wed, Feb 23, 2022 at 12:21 PM bo yang <bobyan...@gmail.com> wrote: >> >>> It is not a standalone spark cluster. In some details, it deploys a >>> Spark Operator ( >>> https://github.com/GoogleCloudPlatform/spark-on-k8s-operator) and an >>> extra REST Service. When people submit Spark application to that REST >>> Service, the REST Service will create a CRD inside the Kubernetes cluster. >>> Then Spark Operator will pick up the CRD and launch the Spark application. >>> The one click tool intends to hide these details, so people could just >>> submit Spark and do not need to deal with too many deployment details. >>> >>> On Tue, Feb 22, 2022 at 8:09 PM Bitfox <bit...@bitfox.top> wrote: >>> >>>> Can it be a cluster installation of spark? or just the standalone node? >>>> >>>> Thanks >>>> >>>> On Wed, Feb 23, 2022 at 12:06 PM bo yang <bobyan...@gmail.com> wrote: >>>> >>>>> Hi Spark Community, >>>>> >>>>> We built an open source tool to deploy and run Spark on Kubernetes >>>>> with a one click command. For example, on AWS, it could automatically >>>>> create an EKS cluster, node group, NGINX ingress, and Spark Operator. Then >>>>> you will be able to use curl or a CLI tool to submit Spark application. >>>>> After the deployment, you could also install Uber Remote Shuffle Service >>>>> to >>>>> enable Dynamic Allocation on Kuberentes. >>>>> >>>>> Anyone interested in using or working together on such a tool? >>>>> >>>>> Thanks, >>>>> Bo >>>>> >>>>>
