Chu Cheng Li created HDDS-16321:
-----------------------------------
Summary: Compute snapshot exclusive size from key visibility
intervals
Key: HDDS-16321
URL: https://issues.apache.org/jira/browse/HDDS-16321
Project: Apache Ozone
Issue Type: Improvement
Reporter: Chu Cheng Li
h2. Summary
Follow-up to HDDS-15125, which records an MVCC visibility interval
({{seqNumMin}} / {{seqNumMax}}) on key versions and uses it to decide
reclaimability without reading the previous snapshot.
Exclusive-size accounting in {{ReclaimableKeyFilter}} still uses the
previous-snapshot lookup path. A deleted key version that *is* referenced by
the previous snapshot therefore still costs a {{getIfExist}} on
{{snapshotRenamedTable}}, a {{get}} on the previous snapshot's {{keyTable}}
including the {{KeyInfo}} parse, and a block-location comparison. That lookup
can be replaced by a single comparison.
h2. Derivation
Exclusive size credits a deleted version to snapshot P iff it is present in P
and absent from P-1:
* present in P: {{seqNumMin <= P.createSeq < seqNumMax}}
* present in P-1: {{seqNumMin <= Pprev.createSeq < seqNumMax}}
HDDS-15125 establishes {{seqNumMax > P.createSeq}} for every entry the filter
sees, because {{OmSnapshotManager.createOmSnapshotCheckpoint}} drains the
bucket's {{deletedTable}} from the active DB as soon as a checkpoint is taken.
Together with {{Pprev.createSeq < P.createSeq}}, "present in P-1" reduces to
{{seqNumMin <= Pprev.createSeq}}. On the branch where the interval already
reported "referenced" we know {{seqNumMin <= P.createSeq}}, so the whole thing
reduces to:
{code:java}
exclusiveToPreviousSnapshot = seqNumMin > previousToPreviousSnapshotCreateIndex;
{code}
No snapshot DB reads on either branch.
h2. Behaviour change to review
For an hsync key whose blocks grew after P was taken, the lookup path treats
the version as absent from P: once the final commit clears {{isHsync}},
{{isBlockLocationInfoSame}} compares block lists and they differ. It therefore
skips the exclusive-size credit, and declares the key reclaimable. The interval
says the version is referenced by P, so it would be credited instead. This
changes reported exclusive sizes for that case.
It is arguably the more self-consistent behaviour, since after HDDS-15125 the
interval already governs the reclaimability verdict, but it is user-visible and
is the reason this is split out of HDDS-15125 rather than shipped with it.
This also needs P-1's {{createTransactionInfo}}. Where it is missing the filter
must decline to the previous-snapshot lookup, as it already does when P's is
missing.
h2. Follow-on cleanup
Once this lands and the interval-less fallback can be retired after upgrade,
{{isReclaimableByPreviousSnapshotLookup}}, {{calculateExclusiveSize}} and
{{getPreviousSnapshotKeyInfo}} can be deleted from {{ReclaimableKeyFilter}} --
roughly 100 lines, including the {{MemoizedCheckedSupplier}} /
{{UncheckedAutoCloseableSupplier}} / {{isBlockLocationInfoSame}} machinery that
exists only to serve them. The filter would then no longer need to open
previous snapshots at all, and {{numberOfPreviousSnapshotsFromChain}} could
drop for the key filter.
That is the point at which HDDS-15125's goal of removing the complex logic from
{{ReclaimableKeyFilter}} is actually realised; HDDS-15125 itself only adds the
interval path alongside the existing one.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
