Hello, Currently I have a Flink 1.4 cluster running on kubernetes based on the configuration describe on https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/deployment/kubernetes.html with additional config for HA with Zookeeper.
With this I have several Taskmanagers, a single Jobmanager and I create a container for each job to perform the Job submission and manage Job updates with savepoints. I'm looking into what would be needed to migrate to the new architecture on FLIP6 as we are planning to use Flink 1.5 once it's ready. If I understand correctly from https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65147077 and the current code on master: * Taskmanagers would continue the same, i.e they will execute the taskmanager.sh start-foreground script, which with the flip6 mode activated will execute the new taskexecutor.TaskManagerRunner. * We will have now one Job Manager per Job which is really good; but I don't fully understand how this would be started. I notice that the jobmanager.sh with flip6 mode activated will execute entrypoint.StandaloneSessionClusterEntrypoint but I don't see how we could pass the job jar and parameters (?) So I think the other possibility to start the job would be via the /flink run/ command with maybe an option to tell that we are creating a job with job manager or would be this the default behaviour ? Or would be this the role of the JobMaster ? I didn't take a look to its code but it's mentioned on the flip6 page. (however I don't see an entrypoint from the scripts (?)) Could you help me to understand how this is expected to be done ? * Also I'm not sure to understand whether it would be better to have a ResourceManager per job or a single ResourceManager per cluster, as in the page is stated that there is a ResourceManager for Self-contained-single-job, but it seems to me that it needs to have the information about all JobManagers and TaskManagers (?) Thanks in advance for the help you could provide. I'm interested in using Flip6 on kubernetes when it will be ready, so I could help with some testing if needed. -- Edward -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
