[
https://issues.apache.org/jira/browse/HDDS-16176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18108816#comment-18108816
]
Chi-Hsuan Huang commented on HDDS-16176:
----------------------------------------
Thanks for confirming! [~echonesis] The scope looks right to me at a glance,
but I'll need to go through the individual charge sites more carefully before
committing to an answer, so I will come back to you.
One simplification: per the discussion on HDDS-16127, multi-version keys aren't
reachable in the current code, and HDDS-15728 will not use multiple location
versions, so versioning doesn't need to be factored in here. And you are right
about the wording: never smaller rather than always larger. I will update the
description.
> EC replicated size is computed per key on commit but per block on delete
> ------------------------------------------------------------------------
>
> Key: HDDS-16176
> URL: https://issues.apache.org/jira/browse/HDDS-16176
> Project: Apache Ozone
> Issue Type: Bug
> Components: OM
> Reporter: Chi-Hsuan Huang
> Priority: Major
>
> h3. Problem
> For an EC key whose data spans more than one block, OM charges quota using a
> different formula than it refunds with. Commit charges
> {{QuotaUtil.getReplicatedSize\(dataSize, repConfig\)}}, converting the whole
> key at once. Delete refunds {{OMKeyRequest.sumBlockLengths\(omKeyInfo\)}},
> converting each block on its own. EC parity is added per block, so the two do
> not agree, and the refund is always the larger of the two.
> {{OmBucketInfo.decrUsedBytes}} does not clamp at zero, so deleting such keys
> can drive a bucket's {{usedBytes}} negative.
> h3. Example
> With {{rs\-3\-2\-1024}}, the data stripe is 3 \* 1024 bytes. A key holding
> two blocks of 1024 bytes each:
> * per key: {{getReplicatedSize\(2048\)}} = 2048 \+ 1024 \* 2 parity = 4096,
> which is what commit charges
> * per block: \(1024 \+ 1024 \* 2 parity\) \* 2 = 6144, which is what delete
> refunds
> h3. Which value describes the stored bytes
> The per\-block value does. EC encodes each block group independently, so each
> block carries its own partial\-stripe padding:
> * {{QuotaUtil.getReplicatedSize}} is documented as "Number of bytes required
> to store the dataSize with replication".
> * {{KeyManagerImpl.java:852\-859}}, which reports reclaimed space to SCM,
> computes both {{getReplicatedSize}} and {{getSizePerReplica}} per block, and
> passes {{OMKeyRequest.sumBlockLengths\(info\)}} as the quota released.
> * {{hadoop\-hdds/docs/content/feature/Quota.md}} describes the space quota as
> the storage space a bucket may use, counted after replication.
> RATIS replication is linear, so the two formulas agree there and only EC is
> affected.
> h3. Also affected: snapshotUsedBytes in quota repair
> {{QuotaRepairTask.recalculateDeletedKeyUsages}}
> \({{QuotaRepairTask.java:513}}\) totals the deleted\-key table with
> {{RepeatedOmKeyInfo.getTotalSize\(\)}}, which converts each key as a whole,
> while {{OMKeyDeleteRequest.java:165}} moves the per\-block
> {{sumBlockLengths\(omKeyInfo\)}} into {{snapshotUsedBytes}}. The same
> divergence therefore applies to that counter, and the formula chosen here
> should be applied there too. That total is wrong for a second, independent
> reason as well, tracked in HDDS\-16184.
> h3. Impact
> Space quota accounting for EC buckets is internally inconsistent. A bucket
> can report a lower {{usedBytes}} than the space its keys occupy, and repeated
> create/delete cycles can push {{usedBytes}} below zero.
> h3. Notes
> Relates to HDDS\-16127, which changes quota repair to recount keys retaining
> more than one version by converting each version once, matching the commit
> path. That patch deliberately does not adopt the per\-block conversion, so
> that repair reproduces what commit charged rather than pre\-empting the
> decision here.
> Pinned source commit 4766aa8609. Analysis assisted by AI tooling \(Claude
> Code, Opus 5\).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
