pnowojski commented on code in PR #28752:
URL: https://github.com/apache/flink/pull/28752#discussion_r3593802241
##########
flink-core/src/main/java/org/apache/flink/configuration/CheckpointingOptions.java:
##########
@@ -611,6 +611,27 @@ public class CheckpointingOptions {
+ " For this feature to be enabled, it
must be also supported by the operator."
+ " Currently this is supported by all
TableStreamOperators and CepOperator.");
+ /**
+ * Controls how often an intermediate watermark is emitted while a
watermark advance is
+ * interrupted before completing (see {@link
#ENABLE_UNALIGNED_INTERRUPTIBLE_TIMERS}). Has no
+ * effect unless interruptible timers are enabled. A value of {@code 0}
disables intermediate
+ * watermark emission entirely.
+ */
+ @Experimental
+ public static final ConfigOption<Duration>
+ UNALIGNED_INTERRUPTIBLE_TIMERS_INTERMEDIATE_WATERMARK_INTERVAL =
+ ConfigOptions.key(
+
"execution.checkpointing.unaligned.interruptible-timers.intermediate-watermark-interval")
+ .durationType()
+ .defaultValue(Duration.ofSeconds(5))
Review Comment:
🤔 Maybe... maybe this should be changed to a boolean flag, and the interval
should taken directly from `pipeline.auto-watermark-interval`?
I've picked 5s default here, because anything > 1s doesn't matter for
performance. `200ms` would have marginal effect if any, hard to say without
benchmarking. However indeed it seems as if re-using here
`pipeline.auto-watermark-interval` makes sense, as it has almsot the same
performance implications vs trying to keep up with real time?
Previously I've thought this features matter only if there is a huge
backpressure, and huge watermark stall. However if we have a lot of fine
grained timers to fire, smaller intervals here can smooth out workload for
downstream operators. For example if job is generally speaking keeping up with
the pending records, but there is an occasional couple of seconds hiccup when a
lot of timers fire.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
