[
https://issues.apache.org/jira/browse/FLINK-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-14162:
-----------------------------------
Labels: pull-request-available (was: )
> Unify SchedulerOperations#allocateSlotsAndDeploy implementation for all
> scheduling strategies
> ---------------------------------------------------------------------------------------------
>
> Key: FLINK-14162
> URL: https://issues.apache.org/jira/browse/FLINK-14162
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Affects Versions: 1.10.0
> Reporter: Zhu Zhu
> Assignee: Zhu Zhu
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> In scheduler NG, scheduling strategies invokes
> {{SchedulerOperations#allocateSlotsAndDeploy(Collection<ExecutionVertexDeploymentOption>)}}
> to trigger scheduling of tasks.
> However, {{EagerSchedulingStrategy}} and
> {{LazyFromSourcesSchedulingStrategy}} both invokes it by passing a batch of
> tasks, but requires the scheduling process to be conducted in 2 different
> ways:
> * {{EagerSchedulingStrategy}} requires the batch of tasks to deploy after
> all of them have acquired slots. This is essential to avoid partition update
> RPCs in streaming job scheduling.
> * {{LazyFromSourcesSchedulingStrategy}} requires tasks in the batch to
> allocate slot and get deployed individually, so that it can deploy a few
> tasks even if the slots is not enough for all tasks in the batch. This is
> helpful for batch job scheduling.
> The scheduler then have to decide the scheduling pattern based whether the
> scheduling strategy is a {{LazyFromSourcesSchedulingStrategy}}. This is not
> good, as there can be more strategies in the future, and even customized
> scheduling strategies.
> I think it's better to define the
> {{SchedulerOperations#allocateSlotsAndDeploy(Collection<ExecutionVertexDeploymentOption>)}}
> to be that all tasks in the batch need to be assigned and deployed together,
> like what we do for {{EagerSchedulingStrategy}}.
> All scheduling strategies need to follow this rule. If tasks should be
> scheduled individually, the strategy should invoke {{allocateSlotsAndDeploy}}
> multiple times, one for each task. As a result, the
> {{LazyFromSourcesSchedulingStrategy}} needs to be adjusted for that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
