Re: Flink on K8s job submission best practices

2018-02-01 Thread Christophe Jolif
Hi Maximilian, Coming back on this as we have similar challenges. I was leaning towards 3. But then I read you and figured I might have missed something ;) We agree 3 is not idiomatic and creates a "detached job" but in a lack of a proper solution I can live with that. We also agree there is no

Re: Flink on K8s job submission best practices

2018-01-03 Thread Robert Metzger
Hi, For the future, the FLIP-6 [1] work will solve the job submission problem in a nice way: You'll be able to build a docker image containing the job and the jobmanager. It's basically a jobmanager configured to only ever run this job. This way, by starting this image, you'll automatically also l

Re: Flink on K8s job submission best practices

2017-12-22 Thread Martin Eden
The above applies to Mesos/DCOS as well. So if someone would also share insights into automatic job deployment in that setup would very useful. Thanks. M On Fri, Dec 22, 2017 at 6:56 PM, Maximilian Bode < maximilian.b...@tngtech.com> wrote: > Hi everyone, > > We are beginning to run Flink on K8s

Flink on K8s job submission best practices

2017-12-22 Thread Maximilian Bode
Hi everyone, We are beginning to run Flink on K8s and found the basic templates [1] as well as the example Helm chart [2] very helpful. Also the discussion about JobManager HA [3] and Patrick's talk [4] was very interesting. All in all it is delightful how easy everything can be set up and works o