Chi-Hsuan Huang created HDDS-16184:
--------------------------------------
Summary: Quota repair undercounts snapshotUsedBytes by collapsing
multi-version keys
Key: HDDS-16184
URL: https://issues.apache.org/jira/browse/HDDS-16184
Project: Apache Ozone
Issue Type: Bug
Components: OM
Reporter: Chi-Hsuan Huang
h3. Problem
Quota repair recounts {{snapshotUsedBytes}} from the deleted\-key table and
reports less than what the delete path moved in, because it collapses a
multi\-version key to its latest version.
h3. Root cause
{{QuotaRepairTask.recalculateDeletedKeyUsages}}
\({{QuotaRepairTask.java:513}}\) totals each deleted\-key entry with
{{RepeatedOmKeyInfo.getTotalSize\(\)}}, which accumulates
{{OmKeyInfo.getReplicatedSize\(\)}} per entry
\({{RepeatedOmKeyInfo.java:117\-128}}\):
{code}
for \(OmKeyInfo omKeyInfo : omKeyInfoList\) {
if \(omKeyInfo.getReplicatedSize\(\) \!= 0\) {
replicatedSize \+= omKeyInfo.getReplicatedSize\(\);
}
unreplicatedSize \+= omKeyInfo.getDataSize\(\);
}
{code}
{{getReplicatedSize\(\)}} is derived from {{dataSize}}, which holds the latest
version only, while the blocks of every retained version are still under
{{OmKeyInfo.getKeyLocationVersions\(\)}}. The delete path instead moves
{{sumBlockLengths\(omKeyInfo\)}} into {{snapshotUsedBytes}}
\({{OMKeyDeleteRequest.java:165,168}}\), which covers all versions.
Deleting a key that retained two versions of 300 and 600 bytes with RATIS ONE
therefore moves 900 bytes into {{snapshotUsedBytes}}, while the recount reports
600 and repair lowers the counter to match.
This is the same defect HDDS\-16127 fixes for {{usedBytes}} on the active key
table, on the other counter and the other table. It does not depend on the
replication type.
h3. Impact
{{snapshotUsedBytes}} is part of {{getTotalBucketSize\(\)}} and is therefore
read by space quota enforcement, so an undercount here relaxes the quota for
buckets with pending deletes. Running repair persists the lower value.
h3. Notes
Separate from HDDS\-16176, which covers a second reason the same total is
wrong: {{getTotalSize\(\)}} converts each key as a whole while the delete path
converts each block, so the two disagree for EC. That question determines which
formula the fix here should use, but the collapse described above is present
regardless of it.
Found while scoping HDDS\-16127, whose PR leaves this path untouched. Pinned
source commit 4766aa8609. Analysis assisted by AI tooling \(Claude Code, Opus
5\).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
