[
https://issues.apache.org/jira/browse/FLINK-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Yao closed FLINK-14362.
----------------------------
Resolution: Fixed
master: f87bc82a8c6f1c1778497fa73bfee11fbf3c750f
> Change DefaultSchedulingResultPartition to return correct partition state
> -------------------------------------------------------------------------
>
> Key: FLINK-14362
> URL: https://issues.apache.org/jira/browse/FLINK-14362
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Affects Versions: 1.10.0
> Reporter: Zhu Zhu
> Assignee: Zhu Zhu
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently {{DefaultSchedulingResultPartition#getState()}} returns the state
> of partitions based on the partition producer's state. The state is used to
> make scheduling decision.
> However, it does not correctly reflect the true state of a partition.
> For example, when producer task turns to RUNNING but has not produced any
> data yet, it's consumers should not be scheduled to reduce unnecessary
> resource cost in lazy scheduling mode. However, the partition state will be
> RUNNING in {{DefaultSchedulingResultPartition}} and will trigger the
> scheduling of its consumers. This may lead to some vertices scheduled earlier
> than expected with no data to consume, which means a waste of resources.
> I'd propose to change the enums in {{ResultPartitionState}} to be:
> * CREATED // partition is just created or is just reset
> * CONSUMABLE // pipelined partition has data produced or blocking partition's
> parent result finishes. Corresponds to
> IntermediateResultPartition#isConsumable.
> The CONSUMABLE state is what the scheduler really cares to make scheduling
> decisions.
> The DefaultResultPartition should be fixed to return {{ResultPartitionState}}
> based the true partition state, rather than the vertex state.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
