I think you are right that I am just seeing "processed in-flight data" and I was misreading the UI.
However, that raises another concern I have. When I deploy a new build that contains changes to the classes for the in-flight data, my deployments have failed when I restore from checkpoints (or savepoints) that show processed bytes. It complains about the serialization version ID which I thought suggested it was trying to restore in-flight data from state to a different version of a class. Thanks! Darin On Mon, Aug 22, 2022 at 10:31 PM yanfei lei <fredia...@gmail.com> wrote: > Hi Darin, > > > I often see my checkpoints contain "Processed (persisted) in-flight > data". > The values outside the parentheses represent Processed in-flight data[1], > and the values inside the parentheses represent persisted in-flight > data[1], what kind of case did you see in your WEB UI？If the >0 values > are outside the parentheses, I think it represents Processed in-flight > data. > > > Is it possible there is some configuration I could be missing? I have > not explicitly disabled unaligned checkpoints but perhaps I should? > You can see whether unaligned checkpoints are enabled by > configuration-tab [2]. > > [1] > https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/ops/monitoring/checkpoint_monitoring/#history-tab > [2] > https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/ops/monitoring/checkpoint_monitoring/#configuration-tab > > > Best, > Yanfei > > Darin Amos via user <user@flink.apache.org> 于2022年8月23日周二 07:02写道： > >> Hi All! >> >> I am running Flink 1.13.1 and I have unaligned checkpoints disabled, >> however I often see my checkpoints contain "Processed (persisted) >> in-flight data". According to the Flink documentation, it states: >> >> - *Persisted in-flight data: The number of bytes persisted during the >> alignment (time between receiving the first and the last checkpoint >> barrier) over all acknowledged subtasks. This is > 0 only if the unaligned >> checkpoints are enabled.* >> >> I'm happy to share screenshots of my Checkpoint configuration and history >> tabs if needed. Colleagues of mine running 1.14.X have also observed the >> same behavior. Is it possible there is some configuration I could be >> missing? I have not explicitly disabled unaligned checkpoints but perhaps I >> should? I wonder if there might be some kind of bug where the UI is showing >> unaligned checkpoints, but the behaviour is not honoring that config. >> >> Thanks! >> >> Darin >> >
