hm.... that'll be better to me if we can build customized resource manager out of core; otherwise, we have to go through the long discussion in the community :) But if we support that, why still mesos/yarn/k8s resource manager there in the tree?
On Fri, Nov 8, 2019 at 10:18 PM Tom Graves <tgraves...@yahoo.com> wrote: > I don't know if it all works but some work was done to make cluster > manager pluggable, see SPARK-13904. > > Tom > > On Wednesday, November 6, 2019, 07:22:59 PM CST, Klaus Ma < > klaus1982...@gmail.com> wrote: > > > Any suggestions? > > - Klaus > > On Mon, Nov 4, 2019 at 5:04 PM Klaus Ma <klaus1982...@gmail.com> wrote: > > Hi team, > > AFAIK, we built k8s/yarn/mesos as resource manager; but I'd like to did > some enhancement to them, e.g. integrate with Volcano > <http://github.com/volcano-sh/volcano> in k8s. Is that possible to do > that without fork the whole spark project? For example, enable customized > resource manager with configuration, e.g. replace > `org.apache.spark.deploy.k8s.submit.KubernetesClientApplication` with > `MyK8SClient`, so I can only maintain the resource manager instead of the > whole project. > > -- Klaus > >
