This is an automated email from the ASF dual-hosted git repository.
junrui pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
from 19bc0b3721e [FLINK-36963][Runtime] Fix wrong context maintain around
`asyncProcessWithKey` (#25855)
new 39ac9e508bd [FLINK-36067][runtime] Introduce StreamGraphOptimizer and
StreamGraphOptimizationStrategy
new 0ce55c37d8c [FLINK-36067][runtime] Manually trigger aggregate
all-to-all result partition info when all consumers created and initialized.
new 95f9a160532 [FLINK-36067][runtime] Support optimize stream graph based
on input info.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../runtime/executiongraph/IntermediateResult.java | 25 ++++
.../executiongraph/IntermediateResultInfo.java | 16 ++-
.../IntermediateResultPartition.java | 2 +-
.../VertexInputInfoComputationUtils.java | 32 +++--
.../runtime/jobgraph/IntermediateDataSet.java | 12 ++
.../adaptivebatch/AbstractBlockingResultInfo.java | 13 +-
.../adaptivebatch/AdaptiveBatchScheduler.java | 89 +++++++++++++-
.../AdaptiveExecutionHandlerFactory.java | 4 +-
.../adaptivebatch/AllToAllBlockingResultInfo.java | 136 ++++++++++++++++-----
.../adaptivebatch/BlockingResultInfo.java | 10 ++
.../DefaultAdaptiveExecutionHandler.java | 48 +++++++-
...faultVertexParallelismAndInputInfosDecider.java | 15 ++-
.../scheduler/adaptivebatch/OperatorsFinished.java | 57 +++++++++
.../adaptivebatch/PointwiseBlockingResultInfo.java | 20 ++-
.../StreamGraphOptimizationStrategy.java | 58 +++++++++
.../adaptivebatch/StreamGraphOptimizer.java | 111 +++++++++++++++++
.../flink/runtime/shuffle/PartitionDescriptor.java | 2 +-
.../streaming/api/graph/AdaptiveGraphManager.java | 7 +-
.../api/graph/DefaultStreamGraphContext.java | 33 ++++-
.../VertexInputInfoComputationUtilsTest.java | 94 ++++++++++----
...AdaptiveExecutionPlanSchedulingContextTest.java | 16 ++-
.../AllToAllBlockingResultInfoTest.java | 27 ++--
.../DefaultAdaptiveExecutionHandlerTest.java | 128 ++++++++++++++++++-
...tVertexParallelismAndInputInfosDeciderTest.java | 102 ++++++++++++----
.../adaptivebatch/StreamGraphOptimizerTest.java | 104 ++++++++++++++++
.../api/graph/DefaultStreamGraphContextTest.java | 6 +-
.../scheduling/AdaptiveBatchSchedulerITCase.java | 112 +++++++++++++++++
27 files changed, 1153 insertions(+), 126 deletions(-)
create mode 100644
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptivebatch/OperatorsFinished.java
create mode 100644
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptivebatch/StreamGraphOptimizationStrategy.java
create mode 100644
flink-runtime/src/main/java/org/apache/flink/runtime/scheduler/adaptivebatch/StreamGraphOptimizer.java
create mode 100644
flink-runtime/src/test/java/org/apache/flink/runtime/scheduler/adaptivebatch/StreamGraphOptimizerTest.java
