Hi! The number of task slots needed is the total number of parallelisms of all the jobs running at the same time (if all the jobs are streaming jobs). The number of task managers is the total number of task slots divide the number of task slots per task manager (taskmanager.numberOfTaskSlots). So it really depends on the jobs you're running.
If you're struggling with determining the number of parallelisms a job needs, try to start with a relatively small number like 8 or 16 and see if that meets your needs, if not then double it and try again. There are also many optimizations which helps you increase the performance if you're using the Table / SQL API, see https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/tuning/ Sigalit Eliazov <[email protected]> 于2021年10月8日周五 下午9:44写道: > > Hello > New to flink and I am struggling with defining the correct configuration. > Currently we decided we have only one job manger. > We have 5 different pipelines. > We are not sure how many task managers needs to be defined, how many slots. > > Can you please send reference or is there any calculator that assist in > such decsions? > > Thanks > > > ---------- Forwarded message --------- > מאת: Sigalit B <[email protected]> > Date: יום ה׳, 7 באוק׳ 2021, 12:40 > Subject: Flink production configuration > To: <[email protected]> > > > Hello > New to flink and I am struggling with defining the correct configuration. > Currently we decided we have only one job manger. > We have 5 different pipelines. > We are not sure how many task managers needs to be defined, how many slots. > > Can you please send reference or is there any calculator that assist in > such decsions? > > Thanks >
