Rui Wang created RATIS-2680:
-------------------------------
Summary: SimpleStateMachineStorage.cleanupOldSnapshots should
consider that consider applications have created legit snapshots without MD5
files
Key: RATIS-2680
URL: https://issues.apache.org/jira/browse/RATIS-2680
Project: Ratis
Issue Type: Improvement
Reporter: Rui Wang
Assignee: Rui Wang
Follow-up to #1320 (RATIS-244), which changed
`SimpleStateMachineStorage.cleanupOldSnapshots` to count only snapshots with
MD5 files toward the retention limit. Meanwhile RATIS-244 maintained the
backward compatibility for `SimpleStateMachineStorage.findLatestSnapshot`.
Overall speaking, there could be applications never considered creating MD5
files for taking snapshots, but because of old implementation of the Ratis,
those applications didn't know that their snapshots should be considered
incomplete or corrupted. Ratis should consider such applications and maintain
backward compatibility. Otherwise once those applications upgrade, they will
surprisingly find the behavior changes of the snapshot.
That change broke backward compatibility during upgrades where old snapshots
(created before MD5 support, without .md5 files) coexist with new snapshots
(with MD5 files) or there is no new snapshots (with MD5 files). In those cases,
cleanup could either skip deletion entirely or retain the wrong set of
snapshots, because old snapshots were excluded from the retention count even
though they were valid before the upgrade.
This PR restores the original retention behavior as a fallback: when fewer than
numSnapshotsRetained MD5 snapshots are found, retain the newest
numSnapshotsRetained snapshots regardless of whether they have MD5 files. This
covers:
1. All old snapshots without MD5 — retention policy applies as it did before
RATIS-244
2. Mixed old and new snapshots after upgrade — old snapshots count toward
retention when there are not enough new MD5 snapshots to fill the quota
3. When enough MD5 snapshots exist, the RATIS-244 behavior is unchanged:
retention is based on MD5 snapshots, and snapshots without MD5 that fall beyond
that cutoff are still cleaned up.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
