Thanks for the answer Patrick! Awesome! I've made a deployment the following way:
Jobmanager is a statefulset with size 1, I'm using EFS(as a persistent volume) and zookeeper for persisting it's state across reboots, the reason it's a statefulset is for having a stable dns for pointing to it "jobmanager-0.jobmanager" TaskManagers are deployed as a Deployment with EFS for persisting checkpoints and snapshots. It's working great so far. Regards 2018-01-24 13:10 GMT-02:00 Patrick Lucas <patr...@data-artisans.com>: > Hi Felipe, > > No, using a Deployment for taskmanagers should be fine—they don't need a > strong identity. For intracluster communication, the taskmanager's hostname > is used by default, which in most Kubernetes setups is resolvable to the Pod > IP. > > state.checkpoints.dir should be configured the same for all jobmanagers and > taskmanagers. All members of the cluster need to be able to see all the > data, and Flink ensures that things are written to the correct subpath. > > -- > Patrick Lucas > > On Mon, Jan 22, 2018 at 7:23 PM, Felipe Cavalcanti <fjfcavalca...@gmail.com> > wrote: >> >> Hi, >> >> I'm deploying flink to kubernetes and I've some doubts... >> >> First one is if the task managers should have strong identity (in >> which case I will use statefulsets for deploying them). Second one is >> if I should point rocksdb state.checkpoint.dir in all task managers to >> the same HDFS path or if each of them should point to their own... >> >> Thanks! > >
