gaoyunhaii commented on a change in pull request #15055:
URL: https://github.com/apache/flink/pull/15055#discussion_r662731089
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RecoveredInputChannel.java
##########
@@ -205,6 +205,12 @@ public void resumeConsumption() {
throw new UnsupportedOperationException("RecoveredInputChannel should
never be blocked.");
}
+ @Override
+ public void acknowledgeAllRecordsProcessed() throws IOException {
+ throw new UnsupportedOperationException(
+ "RecoveredInputChannel should not need acknowledge all records
processed.");
+ }
+
Review comment:
Hi, I think logically this should be supported:
1. Currently the downstream tasks in fact do not consider the
`EndOfUserRecordsEvent` for alignment, the `EndOfUserRecordsEvent` only used to
hold the upstream task till all the records are processed. Then if unaligned
checkpoints are triggered after `EndOfUserRecordsEvent` emitted, the downstream
task could still aligned, and during this process when persisting the channels,
with the current implementation all the events would be ignored, thus the
`EndOfUserRecordsEvent` would not affect the result of the unaligned
checkpoint.
2. Then after failover, with the current implementation the upstream task
would restart and may skip the execution of operators. Then when the new task
finishing, it would re-emitted the `EndOfUserRecordsEvent`. For the recovering
step, `EndOfUserRecordsEvent` should not have impaction since it would not be
included in 1.
Previously we have had ITCase on the PoC branch, but I'm still rebasing and
modifying the following commits, since we would enable the final checkpoint as
a whole at last, thus we might only be able to formally commit the ITCase at
then. Currently the 1 could be covered by the UT and 2 should not happen. I'll
also try to get the following commits done as soon as possible~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
