Hi, Kamal > How many task managers a job manager can handle? Is there any upper limit also?
There is no clear limit to how many TMs a JM can cover, and based on my past experience, it can handle TMs over 1000+, even more. > How to decide no. of task managers, is there any way? I don't think there is a specific formula here, but rather the number of TMs based on your actual business scenario. > What is the difference between high no. of task managers vs high no. of task slots (with low no. of task managers)? Flink uses the SlotSharigGroup mechanism, by default all the operators of the pipeline will run on a Slot, this mechanism will lead to all the operators will share a share of memory with each other, there will be competition, all the Slots of a TM also share the memory of the TM. If there are too many Slots on a TM, they may interfere with each other and affect the stability of the job. If your job requires high stability, it may make more sense to take a high no. of TM way, with fewer Slots per TM. However, too many TMs may lead to too much network transmission overhead, so if the latency requirement of the job is higher, it is more appropriate to adopt the approach of high no. of slot way. Best, Ron Kamal Mittal via user <user@flink.apache.org> 于2023年8月1日周二 14:21写道： > Hell Community, > > > > Need info. for below – > > > > 1. How many task managers a job manager can handle? Is there any upper > limit also? > > > > 1. How to decide no. of task managers, is there any way? > > > > 1. What is the difference between high no. of task managers vs high > no. of task slots (with low no. of task managers)? > > > > Rgds, > > Kamal >
