zhuzhurk commented on a change in pull request #15312:
URL: https://github.com/apache/flink/pull/15312#discussion_r601034351
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/failover/flip1/RestartPipelinedRegionFailoverStrategy.java
##########
@@ -176,15 +183,24 @@ public RestartPipelinedRegionFailoverStrategy(
// if a needed input result partition is not available, its
producer region is involved
for (SchedulingExecutionVertex vertex :
regionToRestart.getVertices()) {
- for (SchedulingResultPartition consumedPartition :
vertex.getConsumedResults()) {
- if (!resultPartitionAvailabilityChecker.isAvailable(
- consumedPartition.getId())) {
- SchedulingPipelinedRegion producerRegion =
- topology.getPipelinedRegionOfVertex(
-
consumedPartition.getProducer().getId());
- if (!visitedRegions.contains(producerRegion)) {
- visitedRegions.add(producerRegion);
- regionsToVisit.add(producerRegion);
+ for (ConsumedPartitionGroup consumedPartitionGroup :
+ vertex.getConsumedPartitionGroups()) {
+ if
(!visitedConsumedResultGroups.contains(consumedPartitionGroup)) {
+
visitedConsumedResultGroups.add(consumedPartitionGroup);
+ for (IntermediateResultPartitionID consumedPartitionId
:
+ consumedPartitionGroup) {
+ if
(!resultPartitionAvailabilityChecker.isAvailable(
Review comment:
maybe extract the finding of unvisited partitions out to avoid such deep
nested loops? e.g.
```
private Iterable<IntermediateResultPartitionID>
getUnvisitedConsumedPartitions(
SchedulingPipelinedRegion region,
Set<ConsumedPartitionGroup> visitedConsumedResultGroups) {
final List<ConsumedPartitionGroup> unvisitedConsumedPartitionGroups
= new ArrayList<>();
for (SchedulingExecutionVertex vertex : region.getVertices()) {
for (ConsumedPartitionGroup consumedPartitionGroup :
vertex.getConsumedPartitionGroups()) {
if
(!visitedConsumedResultGroups.contains(consumedPartitionGroup)) {
continue;
}
visitedConsumedResultGroups.add(consumedPartitionGroup);
unvisitedConsumedPartitionGroups.add(consumedPartitionGroup);
}
}
return () -> IterableUtils.flatMap(unvisitedConsumedPartitionGroups,
Function.identity());
}
```
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/failover/flip1/RestartPipelinedRegionFailoverStrategy.java
##########
@@ -193,13 +209,18 @@ public RestartPipelinedRegionFailoverStrategy(
// all consumer regions of an involved region should be involved
for (SchedulingExecutionVertex vertex :
regionToRestart.getVertices()) {
Review comment:
Similar to above comment, we can introduce a
`getUnvisitedConsumerVertices`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
