Hi! If I should deploy these pipelines in one repository (one application where > I run the multiple services) or deploy each one individually?
This depends on your use case. Running multiple jobs in one deployment is the session mode [1] which shares the job manager and task managers across all jobs. Running each job individually is the per job mode [2] which will create one job manager adn several task managers for each job and should result in better resource isolation but slightly more resource cost. Should I deploy a job manager and task manager for each service or is one > enough for all? As stated above, for session mode one job manager is enough for all submitted jobs, but for per job mode each job needs a separate job manager. As you're running in kubernetes, job managers and task managers should be created automatically when needed. [1] https://nightlies.apache.org/flink/flink-docs-release-1.13/zh/docs/deployment/overview/#session-mode [2] https://nightlies.apache.org/flink/flink-docs-release-1.13/zh/docs/deployment/overview/#per-job-mode Hussein El Ghoul <huss...@quiqup.com> 于2022年1月21日周五 16:27写道： > Dear Community, > > I hope this email finds you well. > I am a beginner and I had some points that needed clarification. > > I am working on multiple flink pipelines, and need to deploy those > pipelines on kubernetes but I am not sure if I should deploy these > pipelines in one repository (one application where I run the multiple > services) or deploy each one individually? In addition, should I deploy a > job manager and task manager for > each service os is one enough for all ? > > Best Regards, > Hussein ElGhoul > Quiqup - Data Engineer
