sjwiesman commented on a change in pull request #12722:
URL: https://github.com/apache/flink/pull/12722#discussion_r444467855
##########
File path: docs/ops/state/checkpoints.md
##########
@@ -113,4 +113,50 @@ above).
$ bin/flink run -s :checkpointMetaDataPath [:runArgs]
{% endhighlight %}
+### Unaligned checkpoints
+
+Starting with Flink 1.11, checkpoints can be unaligned (experimental).
+[Unaligned checkpoints]({% link concepts/stateful-stream-processing.md
+%}#unaligned-checkpointing) contain in-flight data (i.e., data stored in
+buffers) as part of the checkpoint state, which allows checkpoint barriers to
+overtake these buffers. Thus, the checkpoint duration becomes independent of
the
+current throughput as checkpoint barriers are effectively not embedded into
+the stream of data anymore.
+
+You should use unaligned checkpoints if your checkpointing durations are very
+high due to back-pressure. Then, checkpointing time becomes mostly
+independent of the end-to-end latency. Be aware unaligned checkpointing
+adds to I/O to the state backends, so you shouldn't use it when the I/O to
+the state backend is actually the bottleneck during checkpointing.
+
+We flagged unaligned checkpoints as experimental as it currently has the
+following shortcomings:
+
+- You cannot rescale from unaligned checkpoints. You have to take a savepoint
+before rescaling. Savepoints are always aligned independent of the alignment
Review comment:
Yes technically but its incidental. The community hasn't made any
backward compat guaruntees around that behavior.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
