[
https://issues.apache.org/jira/browse/FLINK-25036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lijie Wang updated FLINK-25036:
-------------------------------
Description:
!https://cwiki.apache.org/confluence/download/thumbnails/191334984/Untitled%20%288%29.png?version=1&modificationDate=1634638302000&api=v2!
Considering that the adaptive batch scheduler currently only supports
ALL-EDGES-BLOCKING jobs, and needs to handle the scheduling topology updates
during scheduling, we introduce a new scheduling strategy:
{*}VertexwiseSchedulingStrategy{*}. When an execution vertex is finished, the
new scheduling strategy will try to check the following execution vertices to
see if they can be scheduled :
# All its consumer vertices.
# All execution vertices that are newly added to topology. It is needed
because otherwise some vertices will never be scheduled. e.g. If A1 finishes
first and A2 finishes later, no vertex will be scheduled when A1 finishes
because B1 has not been added to scheduling topology. And only B2 will be
scheduled when A2 finishes, B1 will never be scheduled if we don't try to
schedule the newly added vertices.
was:
Considering adaptive batch scheduler currently only supports ALL-EDGES-BLOCKING
jobs, so we can schedule execution vertices in vertex wise). That means when an
execution vertex is finished, the following execution vertices will be checked
to see if they can be scheduled:
# All its consumer vertices.
# All execution vertices that are newly added to topology. It is needed
because otherwise some vertices will never be scheduled. e.g. If A1 finishes
first and A2 finishes later, no vertex will be scheduled when A1 finishes
because B1 has not been added to scheduling topology. And only B2 will be
scheduled when A2 finishes, B1 will never be scheduled if we don't try to
schedule the newly added vertices.
> Introduce vertex wise scheduling strategy
> -----------------------------------------
>
> Key: FLINK-25036
> URL: https://issues.apache.org/jira/browse/FLINK-25036
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: Lijie Wang
> Assignee: Lijie Wang
> Priority: Major
> Labels: pull-request-available
>
> !https://cwiki.apache.org/confluence/download/thumbnails/191334984/Untitled%20%288%29.png?version=1&modificationDate=1634638302000&api=v2!
>
> Considering that the adaptive batch scheduler currently only supports
> ALL-EDGES-BLOCKING jobs, and needs to handle the scheduling topology updates
> during scheduling, we introduce a new scheduling strategy:
> {*}VertexwiseSchedulingStrategy{*}. When an execution vertex is finished, the
> new scheduling strategy will try to check the following execution vertices to
> see if they can be scheduled :
> # All its consumer vertices.
> # All execution vertices that are newly added to topology. It is needed
> because otherwise some vertices will never be scheduled. e.g. If A1 finishes
> first and A2 finishes later, no vertex will be scheduled when A1 finishes
> because B1 has not been added to scheduling topology. And only B2 will be
> scheduled when A2 finishes, B1 will never be scheduled if we don't try to
> schedule the newly added vertices.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
