[GitHub] [flink] yigress commented on a diff in pull request #23425: [FLINK-33090][checkpointing] CheckpointsCleaner clean individual chec…
yigress commented on code in PR #23425:
URL: https://github.com/apache/flink/pull/23425#discussion_r1336513394
##
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CompletedCheckpoint.java:
##
@@ -368,6 +373,34 @@ public void discard() throws Exception {
}
}
+@Override
+public void discard() throws Exception {
+discard(null);
+}
+
+private void discardOperatorStates(Executor ioExecutor) throws
Exception {
+if (ioExecutor == null) {
+
StateUtil.bestEffortDiscardAllStateObjects(operatorStates.values());
+} else {
+List discardables =
+operatorStates.values().stream()
+.flatMap(op -> op.getDiscardables().stream())
+.collect(Collectors.toList());
+LOG.trace("Executing discard {} operator states {}",
discardables.size());
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] yigress commented on a diff in pull request #23425: [FLINK-33090][checkpointing] CheckpointsCleaner clean individual chec…
yigress commented on code in PR #23425:
URL: https://github.com/apache/flink/pull/23425#discussion_r1336513096
##
flink-core/src/main/java/org/apache/flink/configuration/CheckpointingOptions.java:
##
@@ -109,6 +109,20 @@ public class CheckpointingOptions {
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
+/**
+ * Option whether to clean each checkpoint's states in fast mode. When in
fast mode, operator
+ * states are discarded in parallel using the ExecutorService passed to
the cleaner, otherwise
+ * operator states are discarded sequentially.
+ */
+@Documentation.Section(Documentation.Sections.COMMON_STATE_BACKENDS)
+public static final ConfigOption CLEANER_FAST_MODE =
Review Comment:
updated the config comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] yigress commented on a diff in pull request #23425: [FLINK-33090][checkpointing] CheckpointsCleaner clean individual chec…
yigress commented on code in PR #23425:
URL: https://github.com/apache/flink/pull/23425#discussion_r1336512951
##
flink-core/src/main/java/org/apache/flink/configuration/CheckpointingOptions.java:
##
@@ -109,6 +109,20 @@ public class CheckpointingOptions {
.defaultValue(1)
.withDescription("The maximum number of completed
checkpoints to retain.");
+/**
+ * Option whether to clean each checkpoint's states in fast mode. When in
fast mode, operator
+ * states are discarded in parallel using the ExecutorService passed to
the cleaner, otherwise
+ * operator states are discarded sequentially.
+ */
+@Documentation.Section(Documentation.Sections.COMMON_STATE_BACKENDS)
+public static final ConfigOption CLEANER_FAST_MODE =
Review Comment:
changed to parallel-mode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] yigress commented on a diff in pull request #23425: [FLINK-33090][checkpointing] CheckpointsCleaner clean individual chec…
yigress commented on code in PR #23425:
URL: https://github.com/apache/flink/pull/23425#discussion_r1336512772
##
flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CompletedCheckpoint.java:
##
@@ -368,6 +373,34 @@ public void discard() throws Exception {
}
}
+@Override
+public void discard() throws Exception {
+discard(null);
+}
+
+private void discardOperatorStates(Executor ioExecutor) throws
Exception {
Review Comment:
yes, the logic is the same. Added default method discardOperatorStates in
Checkpoint interface to unify.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] yigress commented on a diff in pull request #23425: [FLINK-33090][checkpointing] CheckpointsCleaner clean individual chec…
yigress commented on code in PR #23425:
URL: https://github.com/apache/flink/pull/23425#discussion_r1336512189
##
flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinatorFailureTest.java:
##
@@ -72,7 +72,7 @@ class CheckpointCoordinatorFailureTest {
/**
* Tests that a failure while storing a completed checkpoint in the
completed checkpoint store
- * will properly fail the originating pending checkpoint and clean upt the
completed checkpoint.
+ * will properly fail the originating pending checkpoint and clean up the
completed checkpoint.
Review Comment:
restore the typo. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
