vitavin commented on code in PR #10585: URL: https://github.com/apache/ignite/pull/10585#discussion_r1143159621
########## docs/_docs/snapshots/snapshots.adoc: ########## @@ -332,6 +366,14 @@ The consistency between the Ignite Persistence files and their snapshot copies i files to the destination snapshot directory with tracking all concurrent ongoing changes. The tracking of the changes might require extra space on the Ignite Persistence storage media (up to the 1x size of the storage media). +=== Incremental snapshot consistency guarantees + +Incremental snapshots uses different non-blocking approach for achieving transactional consistency based on Consistent Cut algorithm. +It allows to start incremental snapshots concurrently with runtime load without affecting performance. But it doesn't guarantee consistency Review Comment: Can we replace “concurrently” with “simultaneously” to be unambiguous? In the case of “concurrently”, this is not clear, we are talking about a simultaneous or parallel process. ########## docs/_docs/snapshots/snapshots.adoc: ########## @@ -76,6 +78,27 @@ snapshot data spread across the cluster. Each node keeps a segment of the snapsh The link:snapshots/snapshots#restoring-from-snapshot[restore procedure] explains how to tether together all the segments during recovery. ==== +== Incremental snapshots + +Low RPO (Recovery Point Object) value like a few minutes can be hardly achieved with full snapshots, since they require additional resources +for creation and storing all partitions data. Instead, incremental snapshots can be used: + +1. store data changes happened since previous full or incremental snapshot +2. consist of compressed WAL segments that are collected in background without pressure on cluster resources +3. provide a lightweight creation process that can be run concurrently with runtime load. + +There are some prerequisites for using incremental snapshots: + +* Incremental snapshots are based on existing full snapshot. +* link:persistence/native-persistence#wal-archive-compaction[WAL archive compaction] has to be enabled. +* Incremental snapshots must be created on the same media drive where WAL archives are stored. + +Restoring procedure consist of multiple steps: at first full snapshot is restored and after that all incremental snapshots Review Comment: Question 1. This sentence is complex and unclear. What does this paragraph mean, an instruction with several steps (at first, we should do…) or just a description of snapshot property? Depending on the meaning, we should create a guide (first, do smth; second, do smth) or to rephrase this sentence in a clear description. Question 2. What does this mean “… are sequentially applied”? If necessary, we should get a replacement for a clear meaning, for example, replace “sequentially” with “consistently”. This will help to emphasize the sequence and connectivity of the process ########## docs/_docs/snapshots/snapshots.adoc: ########## @@ -347,3 +389,15 @@ The snapshot procedure has some limitations that you should be aware of before u If any of these limitations prevent you from using Apache Ignite, then select alternate snapshotting implementations for Ignite provided by enterprise vendors. + +=== Incremental snapshot limitations + +Incremental snapshots can't be created in the following cases: + +* Encrypted caches are presented in a cluster. +* Caches are changed, created or destroyed since full snapshot. Review Comment: It is not clear when the caches change, since the last snapshot was made? It is necessary to clarify, for example, “after a moment when a full snapshot was taken” ########## docs/_docs/snapshots/snapshots.adoc: ########## @@ -332,6 +366,14 @@ The consistency between the Ignite Persistence files and their snapshot copies i files to the destination snapshot directory with tracking all concurrent ongoing changes. The tracking of the changes might require extra space on the Ignite Persistence storage media (up to the 1x size of the storage media). +=== Incremental snapshot consistency guarantees + +Incremental snapshots uses different non-blocking approach for achieving transactional consistency based on Consistent Cut algorithm. Review Comment: What does “based on Consistent Cut Algorithm” mean? Is this property applied to the **approach** or a **consistency**? It seems ambiguous; perhaps, we should paraphrase and clarify this sentence. ########## docs/_docs/snapshots/snapshots.adoc: ########## @@ -332,6 +366,14 @@ The consistency between the Ignite Persistence files and their snapshot copies i files to the destination snapshot directory with tracking all concurrent ongoing changes. The tracking of the changes might require extra space on the Ignite Persistence storage media (up to the 1x size of the storage media). +=== Incremental snapshot consistency guarantees + +Incremental snapshots uses different non-blocking approach for achieving transactional consistency based on Consistent Cut algorithm. +It allows to start incremental snapshots concurrently with runtime load without affecting performance. But it doesn't guarantee consistency +for atomic caches with one or more backups. It's highly recommended to verify these caches after restoring with the `idle_verify` Review Comment: Perhaps we should change to “**It guarantees consistency only for atomic caches that have no backups"** to facilitate a sentence? also may add note ( for atomic caches with backup factor = 0). will it be correct? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
