[
https://issues.apache.org/jira/browse/FLINK-14234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-14234:
----------------------------
Description:
{{SchedulingStrategy}} requires partition consumable notification to make
scheduling decisions.
According to {{SchedulingStrategy#onPartitionConsumable}} definition, all
partition consumable events should be notified to {{SchedulingStrategy}},
including those from TMs (pipelined partitions consumable for data produced)
and from within JM(blocking partitions consumable for producer finished).
In this way, the LazyFromSourcesSchedulingStrategy does not need to maintain
the result partition status by itself. InputDependencyConstraintChecker can be
simplified a lot in this way.
It would also simplify the input checking for pipelined region scheduling.
More details see
[here|https://github.com/apache/flink/pull/9663#discussion_r326540913].
was:
{{SchedulingStrategy}} requires partition consumable notification to make
scheduling decisions.
According to {{SchedulingStrategy#onPartitionConsumable}} definition, all
partition consumable events should be notified to {{SchedulingStrategy}},
including those from TMs (pipelined partitions consumable for data produced)
and from within JM(blocking partitions consumable for producer finished).
More details see
[here|https://github.com/apache/flink/pull/9663#discussion_r326540913].
This can be helpful for a more flexible scheduling.
> All partition consumable events should be notified to SchedulingStrategy
> (SchedulerNG)
> --------------------------------------------------------------------------------------
>
> Key: FLINK-14234
> URL: https://issues.apache.org/jira/browse/FLINK-14234
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Affects Versions: 1.10.0
> Reporter: Zhu Zhu
> Assignee: Zhu Zhu
> Priority: Major
> Fix For: 1.11.0
>
>
> {{SchedulingStrategy}} requires partition consumable notification to make
> scheduling decisions.
> According to {{SchedulingStrategy#onPartitionConsumable}} definition, all
> partition consumable events should be notified to {{SchedulingStrategy}},
> including those from TMs (pipelined partitions consumable for data produced)
> and from within JM(blocking partitions consumable for producer finished).
> In this way, the LazyFromSourcesSchedulingStrategy does not need to maintain
> the result partition status by itself. InputDependencyConstraintChecker can
> be simplified a lot in this way.
> It would also simplify the input checking for pipelined region scheduling.
> More details see
> [here|https://github.com/apache/flink/pull/9663#discussion_r326540913].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
