zentol commented on a change in pull request #8430: [FLINK-12068] [runtime]
Backtrack failover regions if intermediate results are unavailable
URL: https://github.com/apache/flink/pull/8430#discussion_r284214177
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/failover/flip1/RestartPipelinedRegionStrategy.java
##########
@@ -165,6 +181,33 @@ private void buildOneRegionForAllVertices() {
for (FailoverVertex vertex : topology.getFailoverVertices()) {
vertexToRegionMap.put(vertex.getExecutionVertexID(),
region);
}
+
+ buildRegionInputsAndOutputs();
+ }
+
+ private void buildRegionInputsAndOutputs() {
+ for (FailoverRegion region : regions.keySet()) {
+ IdentityHashMap<FailoverRegion, Object> consumers = new
IdentityHashMap<>();
+ Set<IntermediateResultPartitionID> inputs = new
HashSet<>();
+ Set<ExecutionVertexID> consumerVertices = new
HashSet<>();
+ Set<FailoverVertex> regionVertices =
region.getAllExecutionVertices();
+ regionVertices.forEach(v -> {
Review comment:
related: https://github.com/apache/flink/pull/8430#discussion_r284194311
The idea here is fine. I'm wondering if we could merge this logic into the
initial region building; we already iterate over all inputs there anyway. I
think we can even detect region boundaries relatively easy (if an input edge is
not pipelined you have a boundary, and you even know the (temporary) regions)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
