Hey Thanks for trying out GIC! As Yanfei mentioned, GIC and RocksDB incremental do not conflict with each other.
GIC is a generalized way to do incremental Flink checkpoints. At the same time, RocksDB-incremental is a way to do incremental snapshots for backend db (RocksDB). Please let us know the results if convenient :-) Best, Yuan On Sun, Mar 26, 2023 at 2:44 PM Yanfei Lei <fredia...@gmail.com> wrote: > Hi ConradJam, > > > If Generic Incremental Checkpoint (GIC) enable, rocksdb Incremental > Checkpoint can be disable or enable, Do they both have conflicting > switches, does my turning on (GIC) mean I no longer need enable rocksdb > Incremental Checkpoint ? > > The GIC and rocksdb incremental Checkpoint are not in conflict, if > GIC is enabled, rocksdb Incremental Checkpoint can still be enabled > to reduce the amount of data that needs to be uploaded. > GIC is somewhat similar to the concept of WAL in the database, which > can be regarded as a wrapper for original state backends(like heap, > rocksdb, incremental rocksdb). The increment in GIC refers to > **incremental log**, while the increment in the rocksdb state backend > is the incremental SST file related to the specific state backend > implementation. > > Here[1][2] are two blogs about GIC, hope it helps. > > [1] > https://flink.apache.org/2022/05/30/improving-speed-and-stability-of-checkpointing-with-generic-log-based-incremental-checkpoints/ > [2]https://www.ververica.com/blog/generic-log-based-incremental-checkpoint > > > ConradJam <jam.gz...@gmail.com> 于2023年3月26日周日 12:47写道： > > > > Hi Community . I would like to consult about some configurations of > > Rocksdb incremental checkpoints and GIC. In Flink 1.17,I want to try > > this feature . If Generic Incremental Checkpoint (GIC) enable, rocksdb > > Incremental Checkpoint can be disable or enable, Do they both have > > conflicting switches, does my turning on (GIC) mean I no longer need > > enable rocksdb Incremental Checkpoint ? The community seems to have no > > documentation to describe whether the two can be shared or only one of > > them can be enabled, and the other does not need to be enabled > > > > -- > Best, > Yanfei >
