[
https://issues.apache.org/jira/browse/FLINK-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990045#comment-15990045
]
ASF GitHub Bot commented on FLINK-6364:
---------------------------------------
Github user gyfora commented on the issue:
https://github.com/apache/flink/pull/3801
Hi
Thanks for the nice effort!
I only skimmed through the changes to get the main idea (I will do a more
thorough review in the next days) but I have some initial questions :)
1. You mentioned and it looks like that incremental snapshots cannot be
rescaled as it checkpoints the files directly for multiple keygroups together.
How is this constraint enforced? Will the job fail if we try to rescale it?
2. When you restore from a full snapshot will the next incremental snapshot
contain all the files or just the diffs?
3. How do you plan for users to trigger the incremental snapshot operation?
I guess the question goes both for checkpoints and savepoints. For example
every n-th checkpoint is a full snapshot. And users could use flag maybe to
indicate whether the savepoint should be incremental?
Cheers,
Gyula
> Implement incremental checkpointing in RocksDBStateBackend
> ----------------------------------------------------------
>
> Key: FLINK-6364
> URL: https://issues.apache.org/jira/browse/FLINK-6364
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Reporter: Xiaogang Shi
> Assignee: Xiaogang Shi
>
> {{RocksDBStateBackend}} is well suited for incremental checkpointing because
> RocksDB is base on LSM trees, which record updates in new sst files and all
> sst files are immutable. By only materializing those new sst files, we can
> significantly improve the performance of checkpointing.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
