zhuzhurk commented on a change in pull request #8430: [FLINK-12068] [runtime]
Backtrack failover regions if intermediate results are unavailable
URL: https://github.com/apache/flink/pull/8430#discussion_r284684306
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/failover/flip1/RestartPipelinedRegionStrategy.java
##########
@@ -165,6 +181,33 @@ private void buildOneRegionForAllVertices() {
for (FailoverVertex vertex : topology.getFailoverVertices()) {
vertexToRegionMap.put(vertex.getExecutionVertexID(),
region);
}
+
+ buildRegionInputsAndOutputs();
+ }
+
+ private void buildRegionInputsAndOutputs() {
+ for (FailoverRegion region : regions.keySet()) {
+ IdentityHashMap<FailoverRegion, Object> consumers = new
IdentityHashMap<>();
+ Set<IntermediateResultPartitionID> inputs = new
HashSet<>();
+ Set<ExecutionVertexID> consumerVertices = new
HashSet<>();
+ Set<FailoverVertex> regionVertices =
region.getAllExecutionVertices();
+ regionVertices.forEach(v -> {
Review comment:
I tried the merged logic and find the region building becomes even slower.
I think this is due to the merged logic saved some iteration overhead, but
added some more work, including building some legacy relations and later
deprecating them in region merging.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
