[
https://issues.apache.org/jira/browse/HDDS-16127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18105159#comment-18105159
]
Ivan Andika commented on HDDS-16127:
------------------------------------
> One small question: I noticed that
> [setVersioning(true)|https://github.com/apache/ozone/blob/4766aa8609b32ea279c3ea8249e1b835dfa57486/hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java#L366]
> is still reachable, and the [isVersionEnabled
> branch|https://github.com/apache/ozone/blob/4766aa8609b32ea279c3ea8249e1b835dfa57486/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyCommitRequest.java#L328]
> remains in OMKeyCommitRequest, so the Javadoc restriction isn’t enforced.
> Should we address that in HDDS-15879 or HDDS-16014?
Good question, although I think we can discuss it with the author of the new
object versioning instead. I'm not really following the object versioning
feature. I guess the setVersioning should still exist since we might want to
enable bucket versioning (unless the feature will use different flags), but IMO
since there is no difference when versioning is enabled, then we can leave it
be for now. You can add some reviews on these tasks and check with the author
how to handle these (whether they want to handle it in the patch or separately).
The Javadoc should still be valid since I don't think we are use
OmKeyLocationInfoGroup anytime soon. A (very far-fetched) idea I have if in the
future we support multiple HDDS layers (SCMs + DNs) and blocks can be spread
between these two HDDS layer.
I think we can relate all these findings to the HDDS-15728 so that we can
address the legacy code pat.
> Quota repair undercounts usedBytes and usedNamespace for versioning-enabled
> buckets by collapsing multi-version keys
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HDDS-16127
> URL: https://issues.apache.org/jira/browse/HDDS-16127
> Project: Apache Ozone
> Issue Type: Bug
> Components: OM
> Reporter: Siyao Meng
> Assignee: Chi-Hsuan Huang
> Priority: Major
> Labels: pull-request-available
> Attachments: HDDS-16127.001.patch
>
>
> h3. Problem
> For a versioning-enabled bucket, {{ozone admin om quota repair}} undercounts
> {{usedBytes}} and {{usedNamespace}} by collapsing a multi-version key to its
> latest version, then persists the undercount.
> h3. Root cause
> Quota repair recomputes each bucket's usage by scanning the active key table
> and charging, per key-table entry, {{OmKeyInfo.getReplicatedSize()}} (derived
> from the single latest {{dataSize}}) and one namespace unit
> (QuotaRepairTask.java:637-651). For a versioning-enabled bucket, live commit
> accounting instead accumulates every version: each overwrite commit takes the
> versioned branch of {{OMKeyCommitRequest}} that gives the old version no
> credit and adds the new version's replicated size plus one namespace unit,
> while the committed key-table entry keeps {{dataSize}} equal to the latest
> version only. The recount therefore collapses an N-version key to its latest
> version, and repair overwrites the physically correct accumulated count.
> h3. Trigger
> # On a versioning-enabled bucket, commit a key twice (v0 = 300 bytes, v1 =
> 600 bytes) through the normal write path. Live accounting becomes
> usedBytes=900, usedNamespace=2 (both versions retained on disk).
> # Run {{ozone admin om quota repair}}. The active-table scan sees one
> key-table entry with dataSize=600 and recounts usedBytes=600, usedNamespace=1.
> # Repair applies the delta (recount minus live), leaving the bucket at 600
> and 1.
> h3. Impact
> Quota enforcement ({{OMKeyRequest.checkBucketQuotaInBytes}}) and bucket usage
> reporting ({{ozone sh bucket info}}) read the persisted counters. After
> repair writes 600 instead of 900, enforcement under-counts and permits writes
> beyond the bucket's true usage. The undercount is durable, and re-running
> repair reproduces it.
> h3. Suggested fix
> Make the active-table recount version-aware for versioning-enabled buckets:
> sum the replicated size over all {{keyLocationVersions}} groups of each
> {{OmKeyInfo}} and count one namespace unit per version, consistent with
> {{OMKeyCommitRequest}}'s per-commit charge, instead of charging only
> {{getReplicatedSize()}} of the latest {{dataSize}} at
> QuotaRepairTask.java:651. Add a versioned-bucket case to
> {{TestQuotaRepairTask}}.
> h3. Notes
> Discovered via code review under the TLA+ verification effort HDDS-16123. All
> code claims were verified against the source and reproduced with an
> ozone-manager JUnit test using two real {{OMKeyCommitRequest}} commits and
> the real {{QuotaRepairTask}} recount (observed live 900/2, repaired 600/1).
> Distinct from HDDS-15997 / PR 10884, which fixes scan hang and partial counts
> on worker failure and does not touch versioning or {{dataSize}}. Pinned
> source commit 9fbf9ee0cb1bd2f5f5d437b6719ebbe5309351fb. Analysis assisted by
> AI tooling (Claude Code, Opus 4.8) via the Specula pipeline.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
