Re: Running job in detached mode via ClusterClient.

2018-10-02 Thread Till Rohrmann
Hi Piotr, the reason why you cannot submit multiple jobs to a job cluster is that a job cluster is only responsible for a single job. If you want to submit multiple jobs, then you need to start a session cluster. In attached mode, this is currently still possible, because under the hood, we

Running job in detached mode via ClusterClient.

2018-10-02 Thread Piotr Szczepanek
Hey, I've a question regarding submission job in detached mode. We're trying to submit job via YarnClusterClient in detached mode and the first one goes correctly, but when we submit second job we get exception: Multiple environments cannot be created in detached mode at