[
https://issues.apache.org/jira/browse/FLINK-21330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhilong Hong updated FLINK-21330:
---------------------------------
Description:
PipelinedRegionSchedulingStrategy is used for task scheduling. Its
initialization is located at {{PipelinedRegionSchedulingStrategy#init}}. The
initialization can be divided into two parts:
# Calculating consumed result partitions of SchedulingPipelinedRegions
# Calculating the consumer pipelined region of SchedulingResultPartition
Based on FLINK-21328, the {{consumedResults}} of
{{DefaultSchedulingPipelinedRegion}} can be replaced with
{{ConsumedPartitionGroup}}.
Then we can optimize the procedures we mentioned above. After the optimization,
the time complexity decreases from O(N^2) to O(N).
The related usage of {{getConsumedResults}} should be replaced, too.
The detailed design doc is located at:
[https://docs.google.com/document/d/1OjGAyJ9Z6KsxcMtBHr6vbbrwP9xye7CdCtrLvf8dFYw/edit#heading=h.a1mz4yjpry6m]
was:
{{PipelinedRegionSchedulingStrategy}} is used for task scheduling. Its
initialization is located at {{PipelinedRegionSchedulingStrategy#init}}. The
initialization can be divided into two parts:
# Calculating consumed result partitions of SchedulingPipelinedRegions
# Calculating the consumer pipelined region of SchedulingResultPartition
Based on FLINK-21328, the {{consumedResults}} of
{{DefaultSchedulingPipelinedRegion}} can be replaced with
{{ConsumedPartitionGroup}}.
Then we can optimize the procedures we mentioned above. After the optimization,
the time complexity decreases from O(N^2) to O(N).
The related usage of {{getConsumedResults}} should be replaced, too.
The detailed design doc is located at:
[https://docs.google.com/document/d/1OjGAyJ9Z6KsxcMtBHr6vbbrwP9xye7CdCtrLvf8dFYw/edit#heading=h.a1mz4yjpry6m]
> Optimization the initialization of PipelinedRegionSchedulingStrategy
> --------------------------------------------------------------------
>
> Key: FLINK-21330
> URL: https://issues.apache.org/jira/browse/FLINK-21330
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: Zhilong Hong
> Priority: Major
> Fix For: 1.13.0
>
>
> PipelinedRegionSchedulingStrategy is used for task scheduling. Its
> initialization is located at {{PipelinedRegionSchedulingStrategy#init}}. The
> initialization can be divided into two parts:
> # Calculating consumed result partitions of SchedulingPipelinedRegions
> # Calculating the consumer pipelined region of SchedulingResultPartition
> Based on FLINK-21328, the {{consumedResults}} of
> {{DefaultSchedulingPipelinedRegion}} can be replaced with
> {{ConsumedPartitionGroup}}.
> Then we can optimize the procedures we mentioned above. After the
> optimization, the time complexity decreases from O(N^2) to O(N).
> The related usage of {{getConsumedResults}} should be replaced, too.
> The detailed design doc is located at:
> [https://docs.google.com/document/d/1OjGAyJ9Z6KsxcMtBHr6vbbrwP9xye7CdCtrLvf8dFYw/edit#heading=h.a1mz4yjpry6m]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
