Yun Tang created FLINK-25524:
--------------------------------
Summary: If enabled changelog, RocksDB incremental checkpoint
would always be full
Key: FLINK-25524
URL: https://issues.apache.org/jira/browse/FLINK-25524
Project: Flink
Issue Type: Bug
Components: Runtime / Checkpointing, Runtime / State Backends
Reporter: Yun Tang
Once changelog is enabled, RocksDB incremental checkpoint would only be
executed during materialization. During this phase, it will leverage the
{{materization id}} as the checkpoint id for RocksDB state backend's snapshot
method.
However, current incremental checkpoint mechanism heavily depends on the
checkpoint id. And {{SortedMap<Long, Set<StateHandleID>> uploadedStateIDs}}
with checkpoint id as the key within {{RocksIncrementalSnapshotStrategy}} is
the kernel for incremental checkpoint. Once we notify checkpoint complete of
previous checkpoint, it will then remove the uploaded stateIds of that
checkpoint, leading to we cannot get proper checkpoint information on the next
RocksDBKeyedStateBackend#snapshot. That is to say, we will always upload all
RocksDB artifacts.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
