[
https://issues.apache.org/jira/browse/HDDS-17238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-17238:
------------------------------
Affects Version/s: 1.1.0
2.2.1
2.0.0
1.2.1
1.4.0
1.2.0
2.1.2
2.1.0
1.4.1
2.1.1
1.3.0
2.2.0
> Volume create takes refCount and usedNamespace from the request instead of
> initializing them to 0
> -------------------------------------------------------------------------------------------------
>
> Key: HDDS-17238
> URL: https://issues.apache.org/jira/browse/HDDS-17238
> Project: Apache Ozone
> Issue Type: Bug
> Affects Versions: 1.1.0, 1.2.0, 1.3.0, 1.2.1, 1.4.0, 2.0.0, 1.4.1, 2.1.0,
> 2.2.0, 2.1.1, 2.1.2, 2.2.1
> Reporter: Siyao Meng
> Priority: Major
> Attachments:
> CR-4-initialize-volume-refcount-and-used-namespace-on-create.patch
>
>
> h3. Mechanism
> {{usedNamespace}} and {{refCount}} of a volume are maintained by OM: bucket
> create and delete change {{usedNamespace}}, tenant create and delete change
> {{refCount}}. On create, {{OMVolumeCreateRequest.preExecute}} sets only the
> creation and modification time on the request's {{VolumeInfo}}, and
> {{OMVolumeCreateRequest.validateAndUpdateCache}} builds the stored volume
> with {{OmVolumeArgs.builderFromProtobuf}} and then sets objectID, updateID
> and the default ACLs. The two counters are therefore stored with whatever
> value the {{VolumeInfo}} of the request holds, instead of starting at 0.
> {{RpcClient.createVolume}} sets {{usedNamespace}} to 0 and never sets
> {{refCount}}, so both are sent as 0 and volumes created through the Ozone
> client and {{ozone sh}} start at 0 today. Tenant create builds its own
> {{VolumeInfo}} and is not affected.
> h3. Trigger
> A {{VolumeInfo}} that carries {{refCount}} or {{usedNamespace}} reaches
> {{OMVolumeCreateRequest.validateAndUpdateCache}}. Nothing on the OM side
> resets them before the volume is stored.
> h3. Impact
> The new volume starts with counters that do not match its content, and they
> stay in the volume table:
> * With {{refCount}} greater than 0 and no tenant, {{OMVolumeDeleteRequest}}
> rejects the empty volume with {{VOLUME_IS_REFERENCED}}. Only tenant delete
> lowers {{refCount}}.
> * With {{usedNamespace}} equal to {{quotaInNamespace}},
> {{OMBucketCreateRequest}} rejects the first bucket of the empty volume with
> {{QUOTA_EXCEEDED}}.
> Nothing on the OM recomputes either counter of a volume (quota repair
> recomputes bucket counters only).
> h3. Reproduction
> No standalone reproduction is attached; the regression test in the patch
> shows the behaviour. It runs a CreateVolume request with {{refCount=1}} and
> {{usedNamespace=5}} through {{preExecute}} and {{validateAndUpdateCache}}
> and, without the change, finds both values in the stored volume. The two
> effects under Impact were checked with request level tests in
> {{hadoop-ozone/ozone-manager}} on unmodified source: an empty volume stored
> with {{refCount=1}} gets {{VOLUME_IS_REFERENCED}} from
> {{OMVolumeDeleteRequest}}, and one stored with {{quotaInNamespace=5,
> usedNamespace=5}} gets {{QUOTA_EXCEEDED}} for its first bucket. Not run on a
> cluster.
> h3. Patch
> [^CR-4-initialize-volume-refcount-and-used-namespace-on-create.patch],
> against 20d95ed37fe35013b5320713665db03c06ee36fa. It also applies to master
> at 41d95dd3286.
> {{OMVolumeCreateRequest.preExecute}} clears {{usedNamespace}} and
> {{refCount}} in the same builder that sets the creation and modification
> time, so the request written to the Ratis log already carries 0. The apply
> step is unchanged. Requests that already carry 0 behave as before.
> Covered by {{testValidateAndUpdateCacheInitializesRefCountAndUsedNamespace}}
> in the existing {{TestOMVolumeCreateRequest}}, which fails without the change
> ({{[1L, 5L]}} stored). With it the volume request, volume ACL request and
> volume response suites, the tenant create and delete suites and the bucket
> create suites pass (95 tests), checkstyle is clean.
> {{OMBucketCreateRequest}} has the same pattern for the bucket usage counters;
> that is filed separately with its own patch.
> Found by code review of the OM volume, bucket and snapshot lifecycle paths,
> as part of the TLA+ verification effort under HDDS-15926, on commit
> 20d95ed37fe35013b5320713665db03c06ee36fa. Checked against HDDS issues and
> apache/ozone pull requests for duplicates before filing. The attached patch
> is a proposal for review. Generated with Specula (Claude Opus 5.5).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
