[
https://issues.apache.org/jira/browse/HDDS-16184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chi-Hsuan Huang resolved HDDS-16184.
------------------------------------
Resolution: Not A Problem
Not applicable per the discussion on HDDS-16127. No write path produces a key
with more than one location version group, so the collapse cannot happen.
> Quota repair undercounts snapshotUsedBytes by collapsing multi-version keys
> ---------------------------------------------------------------------------
>
> Key: HDDS-16184
> URL: https://issues.apache.org/jira/browse/HDDS-16184
> Project: Apache Ozone
> Issue Type: Bug
> Components: OM
> Reporter: Chi-Hsuan Huang
> Priority: Major
>
> h3. Problem
> Quota repair recounts {{snapshotUsedBytes}} from the deleted\-key table and
> reports less than what the delete path moved in, because it collapses a
> multi\-version key to its latest version.
> h3. Root cause
> {{QuotaRepairTask.recalculateDeletedKeyUsages}}
> \({{QuotaRepairTask.java:513}}\) totals each deleted\-key entry with
> {{RepeatedOmKeyInfo.getTotalSize\(\)}}, which accumulates
> {{OmKeyInfo.getReplicatedSize\(\)}} per entry
> \({{RepeatedOmKeyInfo.java:117\-128}}\):
> {code}
> for \(OmKeyInfo omKeyInfo : omKeyInfoList\) {
> if \(omKeyInfo.getReplicatedSize\(\) \!= 0\) {
> replicatedSize \+= omKeyInfo.getReplicatedSize\(\);
> }
> unreplicatedSize \+= omKeyInfo.getDataSize\(\);
> }
> {code}
> {{getReplicatedSize\(\)}} is derived from {{dataSize}}, which holds the
> latest version only, while the blocks of every retained version are still
> under {{OmKeyInfo.getKeyLocationVersions\(\)}}. The delete path instead moves
> {{sumBlockLengths\(omKeyInfo\)}} into {{snapshotUsedBytes}}
> \({{OMKeyDeleteRequest.java:165,168}}\), which covers all versions.
> Deleting a key that retained two versions of 300 and 600 bytes with RATIS ONE
> therefore moves 900 bytes into {{snapshotUsedBytes}}, while the recount
> reports 600 and repair lowers the counter to match.
> This is the same defect HDDS\-16127 fixes for {{usedBytes}} on the active key
> table, on the other counter and the other table. It does not depend on the
> replication type.
> h3. Impact
> {{snapshotUsedBytes}} is part of {{getTotalBucketSize\(\)}} and is therefore
> read by space quota enforcement, so an undercount here relaxes the quota for
> buckets with pending deletes. Running repair persists the lower value.
> h3. Notes
> Separate from HDDS\-16176, which covers a second reason the same total is
> wrong: {{getTotalSize\(\)}} converts each key as a whole while the delete
> path converts each block, so the two disagree for EC. That question
> determines which formula the fix here should use, but the collapse described
> above is present regardless of it.
> Found while scoping HDDS\-16127, whose PR leaves this path untouched. Pinned
> source commit 4766aa8609. Analysis assisted by AI tooling \(Claude Code, Opus
> 5\).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
