rkhachatryan commented on a change in pull request #16582:
URL: https://github.com/apache/flink/pull/16582#discussion_r678109366
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##########
@@ -1220,8 +1224,15 @@ private void completePendingCheckpoint(PendingCheckpoint
pendingCheckpoint)
Preconditions.checkState(pendingCheckpoint.isDisposed() &&
completedCheckpoint != null);
try {
- completedCheckpointStore.addCheckpoint(
- completedCheckpoint, checkpointsCleaner,
this::scheduleTriggerRequest);
+ CompletedCheckpoint lastSubsumed =
+
completedCheckpointStore.addCheckpointAndSubsumeOldestOne(
+ completedCheckpoint,
+ checkpointsCleaner,
+ this::scheduleTriggerRequest);
+ if (lastSubsumed != null && lastSubsumed.discardOnSubsume()) {
Review comment:
Why do we need `discardOnSubsume` check?
Shouldn't we notify about subsumption regardless of discarding (e.g. even if
it's a savepoint)?
ditto: SubsumeHelper
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##########
@@ -229,6 +229,8 @@
private final ExecutionAttemptMappingProvider attemptMappingProvider;
+ @Nullable private CompletedCheckpoint lastSubsumedCheckpoint;
+
Review comment:
Why do we need this field?
ditto: `lastSubsumedCheckpointId`
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointSubsumeHelper.java
##########
@@ -67,6 +75,7 @@ public static void subsume(
}
// Don't break out from the loop to subsume intermediate savepoints
}
+ return lastSubsumedCheckpoint;
Review comment:
I think it would be more clear if the latest subsumed checkpoint is
recorded by the store in it's subsume action (because of the same ambiguity
about what is returned from this method).
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CompletedCheckpointStore.java
##########
@@ -45,8 +47,12 @@
*
* <p>Only a bounded number of checkpoints is kept. When exceeding the
maximum number of
* retained checkpoints, the oldest one will be discarded.
+ *
+ * @return the subsumed oldest completed checkpoint if possible, return
null if no checkpoint
+ * needs to be discarded on subsume.
*/
- void addCheckpoint(
+ @Nullable
+ CompletedCheckpoint addCheckpointAndSubsumeOldestOne(
Review comment:
The new name doesn't seem to reflect what the method actually does.
It doesn't necessarily subsume **one** checkpoint (or **any** checkpoints at
all).
The correct name would be
`addAndMaybeSubsumeOldestAndReturnTheLatestSubsumedIfAny` - which clearly
indicates to me that the new functionality should be in a separate method.
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java
##########
@@ -1243,6 +1250,29 @@ private void completePendingCheckpoint(PendingCheckpoint
pendingCheckpoint)
CheckpointFailureReason.FINALIZE_CHECKPOINT_FAILURE,
exception);
}
+
+ try {
+
allPreviousCheckpoints.removeAll(completedCheckpointStore.getAllCheckpoints());
+ if (!allPreviousCheckpoints.isEmpty()) {
Review comment:
The new name doesn't seem to reflect what the method actually does.
It doesn't necessarily subsume **one** checkpoint (or **any** checkpoints at
all).
The correct name would be
`addAndMaybeSubsumeOldestAndReturnTheLatestSubsumedIfAny` - which clearly
indicates to me that the new functionality should be in a separate method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
