Siyao Meng created HDDS-17238:
---------------------------------
Summary: Volume create takes refCount and usedNamespace from the
request instead of initializing them to 0
Key: HDDS-17238
URL: https://issues.apache.org/jira/browse/HDDS-17238
Project: Apache Ozone
Issue Type: Bug
Reporter: Siyao Meng
Attachments:
CR-4-initialize-volume-refcount-and-used-namespace-on-create.patch
h3. Mechanism
{{usedNamespace}} and {{refCount}} of a volume are maintained by OM: bucket
create and delete change {{usedNamespace}}, tenant create and delete change
{{refCount}}. On create, {{OMVolumeCreateRequest.preExecute}} sets only the
creation and modification time on the request's {{VolumeInfo}}, and
{{OMVolumeCreateRequest.validateAndUpdateCache}} builds the stored volume with
{{OmVolumeArgs.builderFromProtobuf}} and then sets objectID, updateID and the
default ACLs. The two counters are therefore stored with whatever value the
{{VolumeInfo}} of the request holds, instead of starting at 0.
{{RpcClient.createVolume}} sets {{usedNamespace}} to 0 and never sets
{{refCount}}, so both are sent as 0 and volumes created through the Ozone
client and {{ozone sh}} start at 0 today. Tenant create builds its own
{{VolumeInfo}} and is not affected.
h3. Trigger
A {{VolumeInfo}} that carries {{refCount}} or {{usedNamespace}} reaches
{{OMVolumeCreateRequest.validateAndUpdateCache}}. Nothing on the OM side resets
them before the volume is stored.
h3. Impact
The new volume starts with counters that do not match its content, and they
stay in the volume table:
* With {{refCount}} greater than 0 and no tenant, {{OMVolumeDeleteRequest}}
rejects the empty volume with {{VOLUME_IS_REFERENCED}}. Only tenant delete
lowers {{refCount}}.
* With {{usedNamespace}} equal to {{quotaInNamespace}},
{{OMBucketCreateRequest}} rejects the first bucket of the empty volume with
{{QUOTA_EXCEEDED}}.
Nothing on the OM recomputes either counter of a volume (quota repair
recomputes bucket counters only).
h3. Reproduction
No standalone reproduction is attached; the regression test in the patch shows
the behaviour. It runs a CreateVolume request with {{refCount=1}} and
{{usedNamespace=5}} through {{preExecute}} and {{validateAndUpdateCache}} and,
without the change, finds both values in the stored volume. The two effects
under Impact were checked with request level tests in
{{hadoop-ozone/ozone-manager}} on unmodified source: an empty volume stored
with {{refCount=1}} gets {{VOLUME_IS_REFERENCED}} from
{{OMVolumeDeleteRequest}}, and one stored with {{quotaInNamespace=5,
usedNamespace=5}} gets {{QUOTA_EXCEEDED}} for its first bucket. Not run on a
cluster.
h3. Patch
[^CR-4-initialize-volume-refcount-and-used-namespace-on-create.patch], against
20d95ed37fe35013b5320713665db03c06ee36fa. It also applies to master at
41d95dd3286.
{{OMVolumeCreateRequest.preExecute}} clears {{usedNamespace}} and {{refCount}}
in the same builder that sets the creation and modification time, so the
request written to the Ratis log already carries 0. The apply step is
unchanged. Requests that already carry 0 behave as before.
Covered by {{testValidateAndUpdateCacheInitializesRefCountAndUsedNamespace}} in
the existing {{TestOMVolumeCreateRequest}}, which fails without the change
({{[1L, 5L]}} stored). With it the volume request, volume ACL request and
volume response suites, the tenant create and delete suites and the bucket
create suites pass (95 tests), checkstyle is clean.
{{OMBucketCreateRequest}} has the same pattern for the bucket usage counters;
that is filed separately with its own patch.
Found by code review of the OM volume, bucket and snapshot lifecycle paths, as
part of the TLA+ verification effort under HDDS-15926, on commit
20d95ed37fe35013b5320713665db03c06ee36fa. Checked against HDDS issues and
apache/ozone pull requests for duplicates before filing. The attached patch is
a proposal for review. Generated with Specula (Claude Opus 5.5).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
