[
https://issues.apache.org/jira/browse/FLINK-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374414#comment-16374414
]
Sihua Zhou commented on FLINK-8753:
-----------------------------------
[~aljoscha] Thanks for your reply. I agree that if there is a rough plan for
unifying the format of savepoints between the different state backends then
have a format to normal RocksDB incremental checkpoints would move us very far
from that goal. But I am afraid that unifying the savepoints will lose
performance, which would make the savepoint become useless for some big real
time job. How about introduce to two type of savepoints:
- unified-savepoints (unified between state backends): which is used for job
migration
- backend-savepoints (not unified but performance is excellent): which is used
for job online tunning
What do you think?
> Introduce Incremental savepoint
> -------------------------------
>
> Key: FLINK-8753
> URL: https://issues.apache.org/jira/browse/FLINK-8753
> Project: Flink
> Issue Type: New Feature
> Components: State Backends, Checkpointing
> Affects Versions: 1.5.0
> Reporter: Sihua Zhou
> Assignee: Sihua Zhou
> Priority: Major
>
> Right now, savepoint goes through the full checkpoint path, take a savepoint
> could be slowly. In our production, for some long term job it often costs
> more than 10min to complete a savepoint which is unacceptable for a real time
> job, so we have to turn back to use the externalized checkpoint instead
> currently. But the externalized checkpoint has a time interval (checkpoint
> interval) between the last time. So I proposal to introduce the increment
> savepoint which goes through the increment checkpoint path.
> Any advice would be appreciated!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
