[
https://issues.apache.org/jira/browse/HDDS-16127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18104687#comment-18104687
]
Ivan Andika commented on HDDS-16127:
------------------------------------
Although bucket versioning technically exist in the code by having multiple
versions, there are no code path to exercise it for now, so I don't think it's
within the scope. AFAIK the Ozone versioning HDDS-15728 will not use these
multiple versions location (each version should be a separate OmKeyInfo entry
with a single version). So we can just assume that each key only have one
version. We can add a documentation here.
> Quota repair undercounts usedBytes and usedNamespace for versioning-enabled
> buckets by collapsing multi-version keys
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HDDS-16127
> URL: https://issues.apache.org/jira/browse/HDDS-16127
> Project: Apache Ozone
> Issue Type: Bug
> Components: OM
> Reporter: Siyao Meng
> Assignee: Chi-Hsuan Huang
> Priority: Major
> Labels: pull-request-available
> Attachments: HDDS-16127.001.patch
>
>
> h3. Problem
> For a versioning-enabled bucket, {{ozone admin om quota repair}} undercounts
> {{usedBytes}} and {{usedNamespace}} by collapsing a multi-version key to its
> latest version, then persists the undercount.
> h3. Root cause
> Quota repair recomputes each bucket's usage by scanning the active key table
> and charging, per key-table entry, {{OmKeyInfo.getReplicatedSize()}} (derived
> from the single latest {{dataSize}}) and one namespace unit
> (QuotaRepairTask.java:637-651). For a versioning-enabled bucket, live commit
> accounting instead accumulates every version: each overwrite commit takes the
> versioned branch of {{OMKeyCommitRequest}} that gives the old version no
> credit and adds the new version's replicated size plus one namespace unit,
> while the committed key-table entry keeps {{dataSize}} equal to the latest
> version only. The recount therefore collapses an N-version key to its latest
> version, and repair overwrites the physically correct accumulated count.
> h3. Trigger
> # On a versioning-enabled bucket, commit a key twice (v0 = 300 bytes, v1 =
> 600 bytes) through the normal write path. Live accounting becomes
> usedBytes=900, usedNamespace=2 (both versions retained on disk).
> # Run {{ozone admin om quota repair}}. The active-table scan sees one
> key-table entry with dataSize=600 and recounts usedBytes=600, usedNamespace=1.
> # Repair applies the delta (recount minus live), leaving the bucket at 600
> and 1.
> h3. Impact
> Quota enforcement ({{OMKeyRequest.checkBucketQuotaInBytes}}) and bucket usage
> reporting ({{ozone sh bucket info}}) read the persisted counters. After
> repair writes 600 instead of 900, enforcement under-counts and permits writes
> beyond the bucket's true usage. The undercount is durable, and re-running
> repair reproduces it.
> h3. Suggested fix
> Make the active-table recount version-aware for versioning-enabled buckets:
> sum the replicated size over all {{keyLocationVersions}} groups of each
> {{OmKeyInfo}} and count one namespace unit per version, consistent with
> {{OMKeyCommitRequest}}'s per-commit charge, instead of charging only
> {{getReplicatedSize()}} of the latest {{dataSize}} at
> QuotaRepairTask.java:651. Add a versioned-bucket case to
> {{TestQuotaRepairTask}}.
> h3. Notes
> Discovered via code review under the TLA+ verification effort HDDS-16123. All
> code claims were verified against the source and reproduced with an
> ozone-manager JUnit test using two real {{OMKeyCommitRequest}} commits and
> the real {{QuotaRepairTask}} recount (observed live 900/2, repaired 600/1).
> Distinct from HDDS-15997 / PR 10884, which fixes scan hang and partial counts
> on worker failure and does not touch versioning or {{dataSize}}. Pinned
> source commit 9fbf9ee0cb1bd2f5f5d437b6719ebbe5309351fb. Analysis assisted by
> AI tooling (Claude Code, Opus 4.8) via the Specula pipeline.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
