Re: Migration to Flip6 Kubernetes

2018-05-02 Thread Derek VerLee
Is anyone actively working on direct Kubernetes support? I'd be excited to see this get in sooner rather than later, I'd be happy to start a PR. On 3/22/18 10:37 AM, Till Rohrmann wrote: Hi Edward and Eron, you're

Re: Migration to Flip6 Kubernetes

2018-03-22 Thread Till Rohrmann
Hi Edward and Eron, you're right that there is currently no JobClusterEntrypoint implementation for Kubernetes. How this entrypoint looks like mostly depends on how the job is stored and retrieved. There are multiple ways conceivable: - The entrypoint connects to an external system from which it

Re: Migration to Flip6 Kubernetes

2018-03-21 Thread Edward Rojas
Hi Till, Thanks for the information. We are using the session cluster and is working quite good, but we would like to benefit from the new approach of per-job mode in order to have a better control over the jobs that are running on the cluster. I took a look to the YarnJobClusterEntrypoint and

Re: Migration to Flip6 Kubernetes

2018-03-20 Thread Till Rohrmann
Hi Edward, you're right that Flink's Kubernetes documentation has not been updated with respect to Flip-6. This will be one of the tasks during the Flink 1.5 release testing and is still pending. A Flink cluster can be run in two modes: session mode vs per-job mode. The former starts a cluster

Migration to Flip6 Kubernetes

2018-03-15 Thread Edward Rojas
Hello, Currently I have a Flink 1.4 cluster running on kubernetes based on the configuration describe on https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/deployment/kubernetes.html with additional config for HA with Zookeeper. With this I have several Taskmanagers, a single