Sihua Zhou commented on FLINK-8753:

[~aljoscha] Thanks for your reply. I agree that if there is a rough plan for 
unifying the format of savepoints between the different state backends then 
have a format to normal RocksDB incremental checkpoints would move us very far 
from that goal. But I am afraid that unifying the savepoints will lose 
performance, which would make the savepoint become useless for some big real 
time job. How about introduce to two type of savepoints:

- unified-savepoints (unified between state backends): which is used for job 

- backend-savepoints (not unified but performance is excellent): which is used 
for job online tunning

What do you think?

> Introduce Incremental savepoint
> -------------------------------
>                 Key: FLINK-8753
>                 URL: https://issues.apache.org/jira/browse/FLINK-8753
>             Project: Flink
>          Issue Type: New Feature
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.5.0
>            Reporter: Sihua Zhou
>            Assignee: Sihua Zhou
>            Priority: Major
> Right now, savepoint goes through the full checkpoint path, take a savepoint 
> could be slowly. In our production, for some long term job it often costs 
> more than 10min to complete a savepoint which is unacceptable for a real time 
> job, so we have to turn back to use the externalized checkpoint instead 
> currently. But the externalized  checkpoint has a time interval (checkpoint 
> interval) between the last time. So I proposal to introduce the increment 
> savepoint which goes through the increment checkpoint path.
> Any advice would be appreciated!

This message was sent by Atlassian JIRA

Reply via email to