pnowojski commented on a change in pull request #10332:
[FLINK-13905][checkpointing] Separate checkpoint triggering into several
asynchronous stages
URL: https://github.com/apache/flink/pull/10332#discussion_r363079511
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##########
@@ -957,17 +953,19 @@ private void rememberRecentCheckpointId(long id) {
}
private void dropSubsumedCheckpoints(long checkpointId) {
- Iterator<Map.Entry<Long, PendingCheckpoint>> entries =
pendingCheckpoints.entrySet().iterator();
-
- while (entries.hasNext()) {
- PendingCheckpoint p = entries.next().getValue();
- // remove all pending checkpoints that are lesser than
the current completed checkpoint
- if (p.getCheckpointId() < checkpointId &&
p.canBeSubsumed()) {
- rememberRecentCheckpointId(p.getCheckpointId());
- failPendingCheckpoint(p,
CheckpointFailureReason.CHECKPOINT_SUBSUMED);
- entries.remove();
- }
- }
+ PendingCheckpoint[] checkpointsToSubsume =
+ pendingCheckpoints
+ .values()
+ .stream()
+ .filter(
+ pendingCheckpoint ->
+
pendingCheckpoint.getCheckpointId() < checkpointId &&
+
pendingCheckpoint.canBeSubsumed())
+ .toArray(PendingCheckpoint[]::new);
+
+ abortPendingCheckpoints(
+ checkpointsToSubsume,
+ new
CheckpointException(CheckpointFailureReason.CHECKPOINT_SUBSUMED));
Review comment:
For one thing, you could pass a stream as a parameter to the
`abortPendingCheckpoints`, to deduplicate code a bit. However to be honest I do
not see much value of keeping `abortPendingCheckpoints(PendingCheckpoint[]
checkpoints, CheckpointException exception)` and I think I would just inline it.
Having it, deduplicates only a simple `for (PendingCheckpoint
pendingCheckpoint : checkpoints) ` loop, while callers still have to iterate
over the checkpoints one way or another (creating a stream, filtering it,
collecting to an array).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
