> BTW, from my knowledge, nothing would happen for HashMapStateBackend, which does not support incremental checkpoint yet, when enabling incremental checkpoints.
Thanks Yun, if no errors would occur then definitely +1 to enable it by default Op ma 13 jun. 2022 om 12:42 schreef Alexander Fedulov < alexan...@ververica.com>: > +1 > > From my experience, it is actually hard to come up with use cases where > incremental checkpoints should explicitly not be enabled with the RocksDB > state backend. If the state is so small that the full snapshots do not > have any negative impact, one should consider using HashMapStateBackend > anyway. > > Best, > Alexander Fedulov > > > On Mon, Jun 13, 2022 at 12:26 PM Jing Ge <j...@ververica.com> wrote: > > > +1 > > > > Glad to see the kickoff of this discussion. Thanks Lihe for driving this! > > > > We have actually already discussed it internally a few months ago. After > > considering some corner cases, all agreed on enabling the incremental > > checkpoint as default. > > > > Best regards, > > Jing > > > > On Mon, Jun 13, 2022 at 12:17 PM Yun Tang <myas...@live.com> wrote: > > > > > Strongly +1 for making incremental checkpoints as default. Many users > > have > > > ever been asking why this configuration is not enabled by default. > > > > > > BTW, from my knowledge, nothing would happen for HashMapStateBackend, > > > which does not support incremental checkpoint yet, when enabling > > > incremental checkpoints. > > > > > > > > > Best > > > Yun Tang > > > ________________________________ > > > From: Martijn Visser <martijnvis...@apache.org> > > > Sent: Monday, June 13, 2022 18:05 > > > To: dev@flink.apache.org <dev@flink.apache.org> > > > Subject: Re: [DISCUSS ] Make state.backend.incremental as true by > default > > > > > > Hi Lihe, > > > > > > What happens if we enable incremental checkpoints by default while the > > used > > > memory backend is HashMapStateBackend, which doesn't support > incremental > > > checkpoints? > > > > > > Best regards, > > > > > > Martijn > > > > > > Op ma 13 jun. 2022 om 11:59 schreef Lihe Ma <ma_l...@163.com>: > > > > > > > Hi, Everyone, > > > > > > > > I would like to open a discussion on setting incremental checkpoint > as > > > > default behavior. > > > > > > > > Currently, the configuration of state.backend.incremental is set as > > false > > > > by default. Incremental checkpoint has been adopted widely in > industry > > > > community for many years , and it is also well-tested from the > feedback > > > in > > > > the community discussion. Incremental checkpointing is more > > > light-weighted: > > > > shorter checkpoint duration, less uploaded data and less resource > > > > consumption. > > > > > > > > In terms of backward compatibility, enable incremental checkpointing > > > would > > > > not make any data loss no matter restoring from a full > > > checkpoint/savepoint > > > > or an incremental checkpoint. > > > > > > > > FLIP-193 (Snapshot ownership)[1] has been released in 1.15, > incremental > > > > checkpoint no longer depends on a previous restored checkpoint in > > default > > > > NO_CLAIM mode, which makes the checkpoint lineage much cleaner, it > is a > > > > good chance to change the configuration state.backend.incremental to > > true > > > > as default. > > > > > > > > Thus, based on the above discussion, I suggest to make > > > > state.backend.incremental as true by default. What do you think of > this > > > > proposal? > > > > > > > > [1] > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-193%3A+Snapshots+ownership > > > > > > > > Best regards, > > > > Lihe Ma > > > > > > > > > >
