zhijiangW commented on a change in pull request #11351: [FLINK-16404][runtime]
Solve the potential deadlock problem when reducing exclusive buffers to zero
URL: https://github.com/apache/flink/pull/11351#discussion_r397598435
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/io/CheckpointBarrierAligner.java
##########
@@ -100,21 +105,20 @@ public boolean isBlocked(int channelIndex) {
}
@Override
- public boolean processBarrier(CheckpointBarrier receivedBarrier, int
channelIndex, long bufferedBytes) throws Exception {
+ public void processBarrier(CheckpointBarrier receivedBarrier, int
channelIndex) throws Exception {
final long barrierId = receivedBarrier.getId();
// fast path for single channel cases
if (totalNumberOfInputChannels == 1) {
if (barrierId > currentCheckpointId) {
// new checkpoint
currentCheckpointId = barrierId;
- notifyCheckpoint(receivedBarrier,
bufferedBytes, latestAlignmentDurationNanos);
+ notifyCheckpoint(receivedBarrier,
latestAlignmentDurationNanos);
}
- return false;
+ notifyCheckpointCompletedOrCanceled(barrierId);
Review comment:
There are five paths for calling `notifyCheckpointCompletedOrCanceled` in
this method, then it is a bit hard to trace every path and analysis whether it
needs this call. In contrast, we can analysis this issue from a reverse way to
make it easy. E.g. only two paths (begin new alignment and under current
alignment) do not need the notify call, so we can make a boolean tag only for
these two paths, and handle the unified notification at the end of this method.
In detail, we can define a boolean `shouldNotify = true` at the beginning of
this method and also set the `notifyCheckpointId = currentCheckpointId`. Then
tag this boolean as false in below two paths, and finally call
`notifyCheckpointCompletedOrCanceled` at the end of this method if
`shouldNotify = true`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
