dawidwys commented on a change in pull request #18765: URL: https://github.com/apache/flink/pull/18765#discussion_r827117920
########## File path: docs/content/docs/ops/state/checkpoints_vs_savepoints.md ########## @@ -0,0 +1,97 @@ +--- +title: "Checkpoints VS Savepoints" +weight: 10 +type: docs +aliases: + - /ops/state/checkpoints_vs_savepoints.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Checkpoints VS Savepoints + +## Overview + +Conceptually, Flink's [Savepoints]({{< ref "docs/ops/state/savepoints" >}}) are different from [Checkpoints]({{< ref "docs/ops/state/checkpoints" >}}) +in a similar way that backups are different from recovery logs in traditional database systems. + +The primary purpose of Checkpoints is to provide a recovery mechanism in case of unexpected job failures. +A [Checkpoint's lifecycle]({{< ref "docs/dev/datastream/fault-tolerance/checkpointing" >}}) is managed by Flink, +i.e. a Checkpoint is created, owned, and released by Flink - without user interaction. +As a method of recovery and being periodically triggered, two main design goals for the Checkpoint implementation are +i) being as lightweight to create and ii) being as fast to restore from as possible. +Optimizations towards those goals can exploit certain properties, e.g. that the job code doesn't change between the execution attempts. +Checkpoints are usually dropped after the job was terminated by the user (except if explicitly configured as retained Checkpoints). + +{{< hint info >}} +- Checkpoints are automatically deleted if the application is terminated by the user +(except if checkpoints are explicitly configured to be retained). +- Checkpoints are stored in state backend-specific (native) data format (may be incremental depending on the specific backend). +{{< /hint >}} + +Although [savepoints]({{< ref "docs/ops/state/savepoints" >}}) are created internally with the same mechanisms as +checkpoints, they are conceptually different and can be a bit more expensive to produce and restore from and focus +more on portability and flexibility with respect to changes to the job. +Their use case is planned, manual backup and resume. For example, this could be an update of your Flink version, changing your job graph, +changing parallelism, forking a second job like for a red/blue deployment, and so on. + +{{< hint info >}} +- Savepoints are created, owned and deleted solely by the user. +That means, Flink does not delete savepoints neither after job termination nor after +restore. +- Savepoints are stored in a state backend independent (canonical) format (Note: Since Flink 1.15, savepoints can be also stored in +the backend-specific [native]({{< ref "docs/ops/state/savepoints" >}}#savepoint-format) format which is faster to create +and restore but comes with some limitations. +{{< /hint >}} + +### Capabilities and limitations +The following table gives an overview of capabilities and limitations for the various types of savepoints and +checkpoints. +- ✓ - Flink fully support this type of the snapshot +- x - Flink doesn't support this type of the snapshot +- ! - in fact, Flink support this type but officially, Flink doesn't support this type of the snapshot(there is certain level of risk to use it) + +| Operation | Canonical Savepoint | Native Savepoint | Aligned Checkpoint | Unaligned Checkpoint | +|:--------------------------------|:--------------------|:-----------------|:-------------------|:---------------------| +| State backend change | ✓ | x | x | x | +| State Processor API(writing) | ✓ | x | x | x | +| State Processor API(reading) | ✓ | ! | ! | x | +| Self-contained and relocatable | ✓ | ✓ | x | x | +| Schema evolution | ✓ | ! | ! | ! | Review comment: Haven't we said we want to either update it, or remove it from the current version? @pnowojski -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
