zentol commented on a change in pull request #9250: [FLINK-13371][coordination]
Prevent leaks of blocking partitions
URL: https://github.com/apache/flink/pull/9250#discussion_r309105679
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java
##########
@@ -1066,6 +1066,8 @@ void markFinished(Map<String, Accumulator<?, ?>>
userAccumulators, IOMetrics met
else if (current == CANCELING) {
// we sent a cancel call, and the task manager
finished before it arrived. We
// will never get a CANCELED call back from the
job manager
+ // release all partitions because partitions
should only be kept if the execution reaches FINISHED
+
sendReleaseIntermediateResultPartitionsRpcCall();
Review comment:
> At this point the PartitionTracker is not yet tracking these partitions
(since we never officially reached a state FINISHED in the EG), hence the
execution is sending these through separate RPC logic.
Yes, this is outdated and stems from an earlier version of the PR.
> Note that a similar issue can occur for pipelined partitions that are
buffered in the producers side before a consumer was actually scheduled.
yes, partia...
Emphasis on "similar"; I'm referring to FLINK-13476 here which at the time
did not exist yet.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
