[
https://issues.apache.org/jira/browse/FLINK-31757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711925#comment-17711925
]
Chesnay Schepler commented on FLINK-31757:
------------------------------------------
This scenario can only occur if the user explicitly configured a parallelism of
100 for vertex A and 5 for the remaining vertices.
The obvious solution for the user is to set the parallelism to 100 for
everything if the describe issues are a problem.
In the proposed scenarios of 100 TMs with 1 slot each this is literally the
only option. Just deciding to use p=100 for all vertices ignores an explicit
configuration by the user (bad!), as would mucking around with slot sharing
groups.
> Optimize Flink un-balanced tasks scheduling
> -------------------------------------------
>
> Key: FLINK-31757
> URL: https://issues.apache.org/jira/browse/FLINK-31757
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Reporter: RocMarshal
> Assignee: RocMarshal
> Priority: Major
>
> Supposed we have a Job with 21 {{JobVertex}}. The parallelism of vertex A is
> 100, and the others are 5. If each {{TaskManager}} only have one slot, then
> we need 100 TMs.
> There will be 5 slots with 21 sub-tasks, and the others will only have one
> sub-task of A. Does this mean we have to make a trade-off between wasted
> resources and insufficient resources?
> From a resource utilization point of view, we expect all subtasks to be
> evenly distributed on each TM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
