alpinegizmo commented on a change in pull request #18765: URL: https://github.com/apache/flink/pull/18765#discussion_r833087856
########## File path: docs/content/docs/ops/state/checkpoints_vs_savepoints.md ########## @@ -0,0 +1,96 @@ +--- +title: "Checkpoints VS Savepoints" +weight: 10 +type: docs +aliases: + - /ops/state/checkpoints_vs_savepoints.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Checkpoints VS Savepoints + +## Overview + +Conceptually, Flink's [savepoints]({{< ref "docs/ops/state/savepoints" >}}) are different from [checkpoints]({{< ref "docs/ops/state/checkpoints" >}}) +in a way that's analogous to how backups are different from recovery logs in traditional database systems. + +The primary purpose of checkpoints is to provide a recovery mechanism in case of unexpected job failures. +A [checkpoint's lifecycle]({{< ref "docs/dev/datastream/fault-tolerance/checkpointing" >}}) is managed by Flink, +i.e. a checkpoint is created, owned, and released by Flink - without user interaction. +Because checkpoints are being triggered often, and are relied upon for failure recovery, the two main design goals for the checkpoint implementation are +i) being as lightweight to create and ii) being as fast to restore from as possible. +Optimizations towards those goals can exploit certain properties, e.g., that the job code doesn't change between the execution attempts. +Checkpoints are usually deleted after the job has been terminated by the user (except if explicitly configured as retained checkpoints). + +{{< hint info >}} +- Checkpoints are automatically deleted if the application is terminated by the user +(except if checkpoints are explicitly configured to be retained). +- Checkpoints are stored in state backend-specific (native) data format (may be incremental depending on the specific backend). +{{< /hint >}} + +Although [savepoints]({{< ref "docs/ops/state/savepoints" >}}) are created internally with the same mechanisms as +checkpoints, they are conceptually different and can be a bit more expensive to produce and restore from. Their design focuses +more on portability and operational flexibility, especially with respect to changes to the job. +The use case for savepoints is for planned, manual operations. For example, this could be an update of your Flink version, changing your job graph, and so on. + +{{< hint info >}} +- Savepoints are created, owned and deleted solely by the user. +That means, Flink does not delete savepoints neither after job termination nor after +restore. +- Savepoints are stored in a state backend independent (canonical) format (Note: Since Flink 1.15, savepoints can be also stored in +the backend-specific [native]({{< ref "docs/ops/state/savepoints" >}}#savepoint-format) format which is faster to create +and restore but comes with some limitations. +{{< /hint >}} + +### Capabilities and limitations +The following table gives an overview of capabilities and limitations for the various types of savepoints and +checkpoints. +- ✓ - Flink fully support this type of the snapshot +- x - Flink doesn't support this type of the snapshot +- ! - While these operations currently work, Flink doesn't officially guarantee support for them, so there is a certain level of risk associated with them + +| Operation | Canonical Savepoint | Native Savepoint | Aligned Checkpoint | Unaligned Checkpoint | +|:--------------------------------|:--------------------|:-----------------|:-------------------|:---------------------| +| State backend change | ✓ | x | x | x | +| State Processor API(writing) | ✓ | x | x | x | Review comment: ```suggestion | State Processor API (writing) | ✓ | x | x | x | ``` ########## File path: docs/content/docs/ops/state/checkpoints_vs_savepoints.md ########## @@ -0,0 +1,96 @@ +--- +title: "Checkpoints VS Savepoints" +weight: 10 +type: docs +aliases: + - /ops/state/checkpoints_vs_savepoints.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Checkpoints VS Savepoints + +## Overview + +Conceptually, Flink's [savepoints]({{< ref "docs/ops/state/savepoints" >}}) are different from [checkpoints]({{< ref "docs/ops/state/checkpoints" >}}) +in a way that's analogous to how backups are different from recovery logs in traditional database systems. + +The primary purpose of checkpoints is to provide a recovery mechanism in case of unexpected job failures. +A [checkpoint's lifecycle]({{< ref "docs/dev/datastream/fault-tolerance/checkpointing" >}}) is managed by Flink, +i.e. a checkpoint is created, owned, and released by Flink - without user interaction. +Because checkpoints are being triggered often, and are relied upon for failure recovery, the two main design goals for the checkpoint implementation are +i) being as lightweight to create and ii) being as fast to restore from as possible. +Optimizations towards those goals can exploit certain properties, e.g., that the job code doesn't change between the execution attempts. +Checkpoints are usually deleted after the job has been terminated by the user (except if explicitly configured as retained checkpoints). Review comment: This sentence is redundant with the line below; seems like we don't need it twice. ```suggestion ``` ########## File path: docs/content/docs/ops/state/checkpoints_vs_savepoints.md ########## @@ -0,0 +1,96 @@ +--- +title: "Checkpoints VS Savepoints" Review comment: ```suggestion title: "Checkpoints vs. Savepoints" ``` ########## File path: docs/content/docs/ops/state/checkpoints_vs_savepoints.md ########## @@ -0,0 +1,96 @@ +--- +title: "Checkpoints VS Savepoints" +weight: 10 +type: docs +aliases: + - /ops/state/checkpoints_vs_savepoints.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Checkpoints VS Savepoints + +## Overview + +Conceptually, Flink's [savepoints]({{< ref "docs/ops/state/savepoints" >}}) are different from [checkpoints]({{< ref "docs/ops/state/checkpoints" >}}) +in a way that's analogous to how backups are different from recovery logs in traditional database systems. + +The primary purpose of checkpoints is to provide a recovery mechanism in case of unexpected job failures. +A [checkpoint's lifecycle]({{< ref "docs/dev/datastream/fault-tolerance/checkpointing" >}}) is managed by Flink, +i.e. a checkpoint is created, owned, and released by Flink - without user interaction. +Because checkpoints are being triggered often, and are relied upon for failure recovery, the two main design goals for the checkpoint implementation are +i) being as lightweight to create and ii) being as fast to restore from as possible. +Optimizations towards those goals can exploit certain properties, e.g., that the job code doesn't change between the execution attempts. +Checkpoints are usually deleted after the job has been terminated by the user (except if explicitly configured as retained checkpoints). + +{{< hint info >}} +- Checkpoints are automatically deleted if the application is terminated by the user +(except if checkpoints are explicitly configured to be retained). +- Checkpoints are stored in state backend-specific (native) data format (may be incremental depending on the specific backend). +{{< /hint >}} + +Although [savepoints]({{< ref "docs/ops/state/savepoints" >}}) are created internally with the same mechanisms as +checkpoints, they are conceptually different and can be a bit more expensive to produce and restore from. Their design focuses +more on portability and operational flexibility, especially with respect to changes to the job. +The use case for savepoints is for planned, manual operations. For example, this could be an update of your Flink version, changing your job graph, and so on. + +{{< hint info >}} +- Savepoints are created, owned and deleted solely by the user. +That means, Flink does not delete savepoints neither after job termination nor after +restore. +- Savepoints are stored in a state backend independent (canonical) format (Note: Since Flink 1.15, savepoints can be also stored in +the backend-specific [native]({{< ref "docs/ops/state/savepoints" >}}#savepoint-format) format which is faster to create +and restore but comes with some limitations. +{{< /hint >}} + +### Capabilities and limitations +The following table gives an overview of capabilities and limitations for the various types of savepoints and +checkpoints. +- ✓ - Flink fully support this type of the snapshot +- x - Flink doesn't support this type of the snapshot +- ! - While these operations currently work, Flink doesn't officially guarantee support for them, so there is a certain level of risk associated with them + +| Operation | Canonical Savepoint | Native Savepoint | Aligned Checkpoint | Unaligned Checkpoint | +|:--------------------------------|:--------------------|:-----------------|:-------------------|:---------------------| +| State backend change | ✓ | x | x | x | +| State Processor API(writing) | ✓ | x | x | x | +| State Processor API(reading) | ✓ | ! | ! | x | Review comment: ```suggestion | State Processor API (reading) | ✓ | ! | ! | x | ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
