[
https://issues.apache.org/jira/browse/HDDS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang resolved HDDS-8578.
-----------------------------------
Resolution: Implemented
Resolving as implemented/superseded by HDDS-13730 (Bucket Quota With Snapshots)
and its completed subtasks. The original broad request to investigate namespace
quota during snapshot reclamation is now covered by explicit accounting and
regression tests.
Implementation:
* HDDS-13756 separates active usedNamespace from snapshotUsedNamespace.
Deletion transfers the charge to pending-deletion/snapshot usage rather than
releasing total quota immediately.
* HDDS-13036 uses active plus snapshot usage for quota accounting.
* HDDS-13764 releases pending-deletion namespace through key/directory purge
requests.
* HDDS-13733 protects recreated buckets through bucket-ID checks.
* HDDS-13760 tracks purge transaction flushing to avoid duplicate accounting.
* HDDS-13786 persists bucket-info changes when empty directories are purged.
* HDDS-15650 subsequently added protection for directory delete/purge
snapshotUsedNamespace underflow.
Repository evidence at d0949f291791c399f3e54409fcb92656a80597a4:
* Total bucket namespace and counter transfers:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmBucketInfo.java#L239
* Directory purge accounting:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMDirectoriesPurgeRequestWithFSO.java#L402
* Persistence of updated bucket counters to the active DB, including
snapshot-origin purges:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key/OMDirectoriesPurgeResponseWithFSO.java#L113
Existing test coverage:
* testDirectoryPurge covers eight combinations of active/snapshot DB, parent
purged/retained and empty/populated directories, and verifies namespace
counters after DB batch commit:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMDirectoriesPurgeRequestAndResponse.java#L367
* testSnapshotUsedNamespaceAfterDirectoryDeleteAndPurge checks directory
delete/purge underflow:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/key/TestOMKeyDeleteRequestWithFSO.java#L340
* checkDeletedTableCleanUpForSnapshot checks that snapshot-retained keys stay
charged while reclaimable keys release usage:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/service/TestKeyDeletingService.java#L450
A possible separate follow-up is an end-to-end FSO test covering recursive tree
deletion while a snapshot retains the tree, namespace quota enforcement during
retention, and eventual quota release after snapshot deletion/reclamation. This
is a specific test opportunity rather than a reason to keep this old general
investigation open.
This assessment is based on the implementation and tests present in the
repository; the tests were not rerun during this audit.
> [Snapshot] Check namespace quota handling when cleaning up snapshot
> --------------------------------------------------------------------
>
> Key: HDDS-8578
> URL: https://issues.apache.org/jira/browse/HDDS-8578
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: Snapshot
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
>
> Because FSO namespace quota is only deducted when the directory is purged
> (not when deleted), we need to double check if the quota behavior is as
> expected when doing snapshot reclamation.
> https://github.com/apache/ozone/blob/831f608d28f9f4e040613f2d239a5a2ad109a235/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMDirectoriesPurgeRequestWithFSO.java#L98-L106
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
