Thesharing commented on a change in pull request #16856:
URL: https://github.com/apache/flink/pull/16856#discussion_r692621875
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/DefaultExecutionGraph.java
##########
@@ -1279,10 +1279,14 @@ private void releasePartitions(final
List<IntermediateResultPartitionID> releasa
// Remove cached ShuffleDescriptor when partition is released
releasablePartitions.stream()
-
.map(IntermediateResultPartitionID::getIntermediateDataSetID)
+ .map(edgeManager::getConsumedPartitionGroupsById)
+ .flatMap(List::stream)
.distinct()
- .map(intermediateResults::get)
- .forEach(IntermediateResult::notifyPartitionChanged);
+ .forEach(
+ group ->
+ intermediateResults
+
.get(group.getFirst().getIntermediateDataSetID())
+ .notifyPartitionChanged(group));
Review comment:
Thank you, @tillrohrmann, for this detailed explanation! I think my next
step is to add a hotfix that changes `PartitionReleaseStrategy` to return the
set of completed `ConsumedPartitionGroups`.
Furthermore, a hotfix to get rid of the assumption that one
IntermediateDataSet can have multiple consumer `JobVertices` will also be
added, as we mentioned
[above](https://github.com/apache/flink/pull/16856#discussion_r692056461).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
