Re: How can I set the parallelism higher than the task slot number in more machines?

2020-05-26 Thread Till Rohrmann
Hi Felipe, Flink does not create dummy operators. Unless you have configured one operator to have a parallelism of 32, you should actually only see 16 subtasks of a given operator (given that you start your program with -p 16). Be aware, though, that if you have multiple operators which cannot

Re: How can I set the parallelism higher than the task slot number in more machines?

2020-05-25 Thread Felipe Gutierrez
Solved! that was because I was using slotSharingGroup() in all operators to ensure that they stay in the same task slot. I guess Flink was creating dummy operators to ensure that. Thanks anyway. -- -- Felipe Gutierrez -- skype: felipe.o.gutierrez -- https://felipeogutierrez.blogspot.com On Mon,

How can I set the parallelism higher than the task slot number in more machines?

2020-05-25 Thread Felipe Gutierrez
Hi, I deployed Flink 1.10 standalone in a cluster with 4 machines 8 cores each. Then I configured each machine to have 8 Task Slots and parallelism default of 8. taskmanager.numberOfTaskSlots: 8 parallelism.default: 8 I want to run my stream app with a parallelism of 16 for each subtask. But not