pnowojski commented on a change in pull request #10332:
[FLINK-13905][checkpointing] Separate checkpoint triggering into several
asynchronous stages
URL: https://github.com/apache/flink/pull/10332#discussion_r364209036
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##########
@@ -957,17 +953,19 @@ private void rememberRecentCheckpointId(long id) {
}
private void dropSubsumedCheckpoints(long checkpointId) {
- Iterator<Map.Entry<Long, PendingCheckpoint>> entries =
pendingCheckpoints.entrySet().iterator();
-
- while (entries.hasNext()) {
- PendingCheckpoint p = entries.next().getValue();
- // remove all pending checkpoints that are lesser than
the current completed checkpoint
- if (p.getCheckpointId() < checkpointId &&
p.canBeSubsumed()) {
- rememberRecentCheckpointId(p.getCheckpointId());
- failPendingCheckpoint(p,
CheckpointFailureReason.CHECKPOINT_SUBSUMED);
- entries.remove();
- }
- }
+ PendingCheckpoint[] checkpointsToSubsume =
+ pendingCheckpoints
+ .values()
+ .stream()
+ .filter(
+ pendingCheckpoint ->
+
pendingCheckpoint.getCheckpointId() < checkpointId &&
+
pendingCheckpoint.canBeSubsumed())
+ .toArray(PendingCheckpoint[]::new);
+
+ abortPendingCheckpoints(
+ checkpointsToSubsume,
+ new
CheckpointException(CheckpointFailureReason.CHECKPOINT_SUBSUMED));
Review comment:
Like I mean the call here would be something like:
```
abortPendingCheckpoints(
pendingCheckpoint ->
pendingCheckpoint.getCheckpointId() < checkpointId &&
pendingCheckpoint.canBeSubsumed()),
new CheckpointException(CheckpointFailureReason.CHECKPOINT_SUBSUMED));
```
And the implementation of
`abortPendingCheckpoints(Predicate<PendingCheckpint>
checkpointToDeletePredicate, ...)` could either iterate/remove using the
iterator OR copy the array via streams how you are doing right now - but at
least it (array copy), would be done only in one place, instead of three.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
