Re: Scala program to spark-submit on k8 cluster
If you are looking for a Spark scheduler that runs on top of Kubernetes then this is the way to go: https://github.com/apache/spark/blob/master/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala You can also have a look here: https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html - Thodoris > On 6 Apr 2018, at 15:04, Kittu Mwrote: > > Yes Yinan I’m looking for a Scala program which submits a Spark job to a k8s > cluster by running spark-submit programmatically > >> On Wednesday, April 4, 2018, Yinan Li wrote: >> Hi Kittu, >> >> What do you mean by "a Scala program"? Do you mean a program that submits a >> Spark job to a k8s cluster by running spark-submit programmatically, or some >> example Scala application that is to run on the cluster? >> >>> On Wed, Apr 4, 2018 at 4:45 AM, Kittu M wrote: >>> Hi, >>> >>> I’m looking for a Scala program to spark submit a Scala application (spark >>> 2.3 job) on k8 cluster . >>> >>> Any help would be much appreciated. Thanks >>> >>> >>
Re: Scala program to spark-submit on k8 cluster
Yes Yinan I’m looking for a Scala program which submits a Spark job to a k8s cluster by running spark-submit programmatically On Wednesday, April 4, 2018, Yinan Liwrote: > Hi Kittu, > > What do you mean by "a Scala program"? Do you mean a program that submits > a Spark job to a k8s cluster by running spark-submit programmatically, or > some example Scala application that is to run on the cluster? > > On Wed, Apr 4, 2018 at 4:45 AM, Kittu M wrote: > >> Hi, >> >> I’m looking for a Scala program to spark submit a Scala application >> (spark 2.3 job) on k8 cluster . >> >> Any help would be much appreciated. Thanks >> >> >> >
Re: Scala program to spark-submit on k8 cluster
yes “REST application that submits a Spark job to a k8s cluster by running spark-submit programmatically” and also would like to expose as a Kubernetes service so that clients can access as any other Rest api On Wed, Apr 4, 2018 at 12:25 PM Yinan Liwrote: > Hi Kittu, > > What do you mean by "a Scala program"? Do you mean a program that submits > a Spark job to a k8s cluster by running spark-submit programmatically, or > some example Scala application that is to run on the cluster? > > On Wed, Apr 4, 2018 at 4:45 AM, Kittu M wrote: > >> Hi, >> >> I’m looking for a Scala program to spark submit a Scala application >> (spark 2.3 job) on k8 cluster . >> >> Any help would be much appreciated. Thanks >> >> >> >
Re: Scala program to spark-submit on k8 cluster
Hi Kittu, What do you mean by "a Scala program"? Do you mean a program that submits a Spark job to a k8s cluster by running spark-submit programmatically, or some example Scala application that is to run on the cluster? On Wed, Apr 4, 2018 at 4:45 AM, Kittu Mwrote: > Hi, > > I’m looking for a Scala program to spark submit a Scala application (spark > 2.3 job) on k8 cluster . > > Any help would be much appreciated. Thanks > > >