Chi-Hsuan Huang created HDDS-16178:
--------------------------------------
Summary: Allocated blocks can survive under a stale createVersion
after a versioned overwrite
Key: HDDS-16178
URL: https://issues.apache.org/jira/browse/HDDS-16178
Project: Apache Ozone
Issue Type: Bug
Components: OM
Reporter: Chi-Hsuan Huang
h3. Problem
On a versioning\-enabled overwrite, blocks that were allocated but never
committed can stay attached to the committed key under a stale
{{createVersion}}. Every caller that flattens a key's location groups, most
importantly {{OMKeyRequest.sumBlockLengths}}, then counts those blocks at their
full allocated length in addition to the committed ones.
h3. Root cause
Blocks handed out by {{allocateBlock}} carry the default {{createVersion}} of 0
and a length of the SCM block size. On an overwrite of a versioning\-enabled
bucket, {{OMKeyRequest.prepareFileInfo}} calls {{addNewVersion\(locations,
false, true\)}}, and {{OmKeyLocationInfoGroup.generateNextVersion}}
\({{OmKeyLocationInfoGroup.java:160\-165}}\) files them under map key {{version
\+ 1}} while the block objects still report {{createVersion=0}}:
{code}
OmKeyLocationInfoGroup generateNextVersion\(List newLocationList\) {
Map> newMap = new HashMap<>\(\);
newMap.put\(version \+ 1, new ArrayList<>\(newLocationList\)\);
return new OmKeyLocationInfoGroup\(version \+ 1, newMap\);
}
{code}
If the open key is written to RocksDB and read back before the commit is
processed, {{OmKeyLocationInfoGroup.getFromProtobuf}}
\({{OmKeyLocationInfoGroup.java:139\-148}}\) regroups the entries by their
{{createVersion}}, so the new group's internal map becomes {{{0: \[allocated
blocks\], 1: \[\]}}}, the empty entry coming from the constructor's
{{putIfAbsent}}.
Commit then only touches the latest key. {{OmKeyInfo.updateLocationInfoList}}
\({{OmKeyInfo.java:330\-333}}\) runs
{{keyLocationInfoGroup.removeBlocks\(latestVersion\)}} and
{{addAll\(latestVersion, updatedBlockLocations\)}}, which clears and
repopulates map key 1 and leaves the entries under map key 0 untouched.
{{getLocationList\(\)}} flattens the whole map, so those stale entries are
returned alongside the committed blocks.
{{OMAllocateBlockRequest}} is not affected, because it appends through
{{appendNewBlocks}}, which sets {{createVersion}} on each block.
h3. Evidence
The regrouping was reproduced in an ozone\-manager unit test. An open key built
as an overwrite whose new blocks carry {{createVersion=0}}, written to the open
key table and read back, has groups {{\[v0\[0\], v1\[0, 1\]\]}}: the block
intended for version 1 sits under map key 0 of the version 1 group.
The survival of those entries through commit follows from
{{OmKeyInfo.java:330\-333}} but has not been reproduced end to end. An
integration test using the real client, with the open key flushed before
commit, would confirm it.
h3. Impact
Any consumer of the flattened block list over\-counts.
{{OMKeyRequest.sumBlockLengths}} is used to release quota in
{{OMKeyDeleteRequest}}, {{OMKeysDeleteRequest}}, the FSO variants,
{{OMDirectoriesPurgeRequestWithFSO}} and {{KeyManagerImpl}}, so a delete can
refund more than was charged. After HDDS\-16127 the quota repair recount also
uses it for versioning\-enabled buckets.
h3. Notes
Found while scoping HDDS\-16127. Pinned source commit 4766aa8609. Analysis
assisted by AI tooling \(Claude Code, Opus 5\).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
