Chi-Hsuan Huang created HDDS-16176:
--------------------------------------
Summary: EC replicated size is computed per key on commit but per
block on delete
Key: HDDS-16176
URL: https://issues.apache.org/jira/browse/HDDS-16176
Project: Apache Ozone
Issue Type: Bug
Components: OM
Reporter: Chi-Hsuan Huang
h3. Problem
For an EC key whose data spans more than one block, OM charges quota using a
different formula than it refunds with. Commit charges
{{QuotaUtil.getReplicatedSize\(dataSize, repConfig\)}}, converting the whole
key at once. Delete refunds {{OMKeyRequest.sumBlockLengths\(omKeyInfo\)}},
converting each block on its own. EC parity is added per block, so the two do
not agree, and the refund is always the larger of the two.
{{OmBucketInfo.decrUsedBytes}} does not clamp at zero, so deleting such keys
can drive a bucket's {{usedBytes}} negative.
h3. Example
With {{rs\-3\-2\-1024}}, the data stripe is 3 \* 1024 bytes. A key holding two
blocks of 1024 bytes each:
* per key: {{getReplicatedSize\(2048\)}} = 2048 \+ 1024 \* 2 parity = 4096,
which is what commit charges
* per block: \(1024 \+ 1024 \* 2 parity\) \* 2 = 6144, which is what delete
refunds
h3. Which value describes the stored bytes
The per\-block value does. EC encodes each block group independently, so each
block carries its own partial\-stripe padding:
* {{QuotaUtil.getReplicatedSize}} is documented as "Number of bytes required to
store the dataSize with replication".
* {{KeyManagerImpl.java:852\-859}}, which reports reclaimed space to SCM,
computes both {{getReplicatedSize}} and {{getSizePerReplica}} per block, and
passes {{OMKeyRequest.sumBlockLengths\(info\)}} as the quota released.
* {{hadoop\-hdds/docs/content/feature/Quota.md}} describes the space quota as
the storage space a bucket may use, counted after replication.
RATIS replication is linear, so the two formulas agree there and only EC is
affected.
h3. Also affected: snapshotUsedBytes in quota repair
{{QuotaRepairTask.recalculateDeletedKeyUsages}}
\({{QuotaRepairTask.java:513}}\) totals the deleted\-key table with
{{RepeatedOmKeyInfo.getTotalSize\(\)}}, which sums
{{OmKeyInfo.getReplicatedSize\(\)}}, i.e. the latest version's {{dataSize}}
converted as a whole. {{OMKeyDeleteRequest.java:165}} moves
{{sumBlockLengths\(omKeyInfo\)}}, covering all versions and converted per
block, into {{snapshotUsedBytes}}. The recount therefore reports less than what
was moved in, for the same two reasons: it collapses multi\-version keys and it
converts per key.
h3. Impact
Space quota accounting for EC buckets is internally inconsistent. A bucket can
report a lower {{usedBytes}} than the space its keys occupy, and repeated
create/delete cycles can push {{usedBytes}} below zero.
h3. Notes
Relates to HDDS\-16127, which changes quota repair to recount
versioning\-enabled buckets with {{sumBlockLengths}}. The non\-versioned
recount path there still converts per key, so the outcome of this issue
determines whether both paths converge.
Pinned source commit 4766aa8609. Analysis assisted by AI tooling \(Claude Code,
Opus 5\).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
