Hi community, I'm running an application to consume data from kafka, and process it then put data to the druid. I wonder if there is a way where I can allocate the data source consuming process evenly across the task manager to maximize the usage of the network of task managers.
So, for example, I have 15 task managers and I set parallelism for the kafka source as 60, since I have 60 partitions in kafka topic. What I want is flink cluster will put 4 kafka source subtasks on each task manager. Is that possible? I have gone through the document, the only thing we found is cluster.evenly-spread-out-slots which does exact the opposite of what I want. It will put the subtasks of the same operator onto one task manager as much as possible. So, is some kind of manual resource allocation available? Thanks in advance! Best regards, Mu
