RE: External shuffle service on K8S

2018-10-27 Thread Garlapati, Suryanarayana (Nokia - IN/Bangalore)
Cheah Sent: Saturday, October 27, 2018 6:12 AM To: Li Gao ; vincent.gromakow...@gmail.com Cc: caolijun1...@gmail.com; user@spark.apache.org Subject: Re: External shuffle service on K8S Hi there, Please see https://issues.apache.org/jira/browse/SPARK-25299 for more discussion around this matter

Re: External shuffle service on K8S

2018-10-26 Thread Matt Cheah
apache.org" Subject: Re: External shuffle service on K8S There are existing 2.2 based ext shuffle on the fork: https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html [apache-spark-on-k8s.github.io] You can modify it to suit your needs. -Li On Fri, O

Re: External shuffle service on K8S

2018-10-26 Thread Li Gao
There are existing 2.2 based ext shuffle on the fork: https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html You can modify it to suit your needs. -Li On Fri, Oct 26, 2018 at 3:22 AM vincent gromakowski < vincent.gromakow...@gmail.com> wrote: > No it's on the roadmap >2.4 >

Re: External shuffle service on K8S

2018-10-26 Thread vincent gromakowski
No it's on the roadmap >2.4 Le ven. 26 oct. 2018 à 11:15, 曹礼俊 a écrit : > Hi all: > > Does Spark 2.3.2 supports external shuffle service on Kubernetes? > > I have looked up the documentation( > https://spark.apache.org/docs/latest/running-on-kubernetes.html), but > couldn't find related