Hi Kyle, So you have a static Dask cluster running your k8s cluster? Is there any reason you wouldn't just launch the Dask cluster for the job you're running and then tear it down? I feel like with k8s the elasticity is one of the main benefits.
On Fri, Apr 27, 2018 at 12:32 PM Kyle Hamlin <[email protected]> wrote: > Hi all, > > If I have a Kubernetes cluster running in DCOC and a Dask cluster running > in that same Kubernetes cluster is it possible/does it makes sense to use > the KubernetesExecutor to launch tasks into the Dask cluster (these are ML > jobs with sklearn)? I feel like there is a bit of inception going on here > in my mind and I just want to make sure a setup like this makes sense? > Thanks in advance for anyone's input! >
