rkhachatryan commented on a change in pull request #15728:
URL: https://github.com/apache/flink/pull/15728#discussion_r621050759
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/DefaultCheckpointPlanCalculator.java
##########
@@ -111,7 +112,10 @@ public void setAllowCheckpointsAfterTasksFinished(boolean
allowCheckpointsAfterT
? calculateAfterTasksFinished()
: calculateWithAllTasksRunning();
- checkTasksStarted(result.getTasksToTrigger());
+ checkTasksStarted(
+ isUnalignedCheckpoint
+ ? result.getTasksToWaitFor()
+ : result.getTasksToTrigger());
Review comment:
What @AHeise described makes sense to me (skipping processing of data
accumulated during recovery and passing barriers right away, reducing e2e
latency) but it seems quite a narrow use-case:
- checkpoint timeout must be less than recovery time
- time to process the accumulated data must be significantly higher than
checkpoint interval
So one alternative would be to add an option to allow power-users to trigger
checkpoints while still in recovery, but don't trigger by default. But such
option can also be added separately.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
