[
https://issues.apache.org/jira/browse/RATIS-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz-wo Sze updated RATIS-2680:
------------------------------
Component/s: snapshot
> SimpleStateMachineStorage.cleanupOldSnapshots should consider that consider
> applications have created legit snapshots without MD5 files
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: RATIS-2680
> URL: https://issues.apache.org/jira/browse/RATIS-2680
> Project: Ratis
> Issue Type: Improvement
> Components: snapshot
> Reporter: Rui Wang
> Assignee: Rui Wang
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Follow-up to #1320 (RATIS-244), which changed
> `SimpleStateMachineStorage.cleanupOldSnapshots` to count only snapshots with
> MD5 files toward the retention limit. Meanwhile RATIS-244 maintained the
> backward compatibility for `SimpleStateMachineStorage.findLatestSnapshot`.
> Overall speaking, there could be applications never considered creating MD5
> files for taking snapshots, but because of old implementation of the Ratis,
> those applications didn't know that their snapshots should be considered
> incomplete or corrupted. Ratis should consider such applications and maintain
> backward compatibility. Otherwise once those applications upgrade, they will
> surprisingly find the behavior changes of the snapshot.
> That change broke backward compatibility during upgrades where old snapshots
> (created before MD5 support, without .md5 files) coexist with new snapshots
> (with MD5 files) or there is no new snapshots (with MD5 files). In those
> cases, cleanup could either skip deletion entirely or retain the wrong set of
> snapshots, because old snapshots were excluded from the retention count even
> though they were valid before the upgrade.
> This PR restores the original retention behavior as a fallback: when fewer
> than numSnapshotsRetained MD5 snapshots are found, retain the newest
> numSnapshotsRetained snapshots regardless of whether they have MD5 files.
> This covers:
> 1. All old snapshots without MD5 — retention policy applies as it did before
> RATIS-244
> 2. Mixed old and new snapshots after upgrade — old snapshots count toward
> retention when there are not enough new MD5 snapshots to fill the quota
> 3. When enough MD5 snapshots exist, the RATIS-244 behavior is unchanged:
> retention is based on MD5 snapshots, and snapshots without MD5 that fall
> beyond that cutoff are still cleaned up.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
