[
https://issues.apache.org/jira/browse/FLINK-31757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711776#comment-17711776
]
RocMarshal edited comment on FLINK-31757 at 4/13/23 9:24 AM:
-------------------------------------------------------------
h1. Problem description and impact
h1. The case
Supposed a Job has 21 tasks:
* Task A has the parallelism of 100,
* The every remained task has the parallelism of 5.
Each TM slot = 1, so the tasks in the job need to apply for 100 TMs.
h2. Problem Description
Assuming that the TM number is 0-99, from the perspective of Task, the actual
result after scheduling is:
After the job deployed. There are 5 TMs loading with 21 sub-tasks, while other
TMs only load a sub-task.
h2. Influence
If the user allocates resources to TM: All TM resources are applied according
to the 5 TMs (loading 21-subtasks), then subsequent TM resources will be
wasted. If apply the resources based on other TM(only loading a subtask), the 5
TMs resources are insufficient, tasks running on its may have lag.
>From the perspective of resource usage, we expect all subtasks to be evenly
>distributed on each TM.
was (Author: rocmarshal):
h1. Problem description and impact
Supposed a Job has 21 tasks:
* Task A has the parallelism of 100,
* The every remained task has the parallelism of 5.
Each TM slot = 1, so the tasks in the job need to apply for 100 TMs.
h2. Problem Description
Assuming that the TM number is 0-99, from the perspective of Task, the actual
result after scheduling is:
After the job deployed. There are 5 TMs loading with 21 sub-tasks, while other
TMs only load a sub-task.
h2. Influence
If the user allocates resources to TM: All TM resources are applied according
to the 5 TMs (loading 21-subtasks), then subsequent TM resources will be
wasted. If apply the resources based on other TM(only loading a subtask), the 5
TMs resources are insufficient, tasks running on its may have lag.
>From the perspective of resource usage, we expect all subtasks to be evenly
>distributed on each TM.
> Optimize Flink un-balanced tasks scheduling
> -------------------------------------------
>
> Key: FLINK-31757
> URL: https://issues.apache.org/jira/browse/FLINK-31757
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Task
> Reporter: RocMarshal
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
