[
https://issues.apache.org/jira/browse/HDDS-13188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanvi Penumudy updated HDDS-13188:
----------------------------------
Description:
The PR introduces a *GET* /api/v1/keys/deletePending/dirs/size endpoint to
include _totalUnreplicatedDataSize_ and _totalReplicatedDataSize_ fields for
tracking the total pending deletion size for FSO directories in OM.
* {_}totalReplicatedDataSize{_}: Total size of directories pending deletion
with expected replicated size.
* {_}totalUnreplicatedDataSize{_}: Total raw size of directories pending
deletion without replicated size.
was:
Implement logic to capture FSO/OBS delete events and track pending bytes in an
external tracking structure:
OBS → The deleted entries can be obtained from the OM DB deletedTable directly.
FSO → Recon can help track deleted directory sizes through the Namespace
Summary (NSSummaryTree), outlined below:
When a directory is deleted:
The NSSummaryTree receives a delete directory event as part of OM.DB delta
updates.
This event is captured via the NSSummaryTaskDbEventHandler#handleDeleteDirEvent
method.
The delete directory event can be intercepted immediately upon capture prior to
the NSSummaryTree being updated through the above method.
Recon allows retrieval of the NSSummary object corresponding to the deleted
directory’s objectID. The NSSummary object includes the getSizeOfFiles() value,
representing the total size of files under the directory.
At the time of capturing the delete event, this total calculated size can be
stored in an external table (in Recon’s Derby DB) against the deleted directory
object ID.
In the future, as and when we receive the individual file delete events
(captured via NSSummaryTaskDbEventHandler#handleDeleteKeyEvent), we do the
following:
Check if the file belonged to a deleted directory.
Subtract the file's size from the corresponding deleted directory entry in the
external table.
Once the deleted size for a directory becomes zero, the entry is removed from
the external table.
The NSSummaryTree is persisted (in RocksDB) and so is the data under the Derby
DB. Hence the data remains intact even when Recon restarts. At any time, we can
iterate over this external table to compute the total size of FSO deleted keys
still pending movement to the OM DB deletedTable.
> Track Pending Deletion (OM)
> ---------------------------
>
> Key: HDDS-13188
> URL: https://issues.apache.org/jira/browse/HDDS-13188
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Priyesh K
> Assignee: Tanvi Penumudy
> Priority: Major
> Labels: pull-request-available
>
> The PR introduces a *GET* /api/v1/keys/deletePending/dirs/size endpoint to
> include _totalUnreplicatedDataSize_ and _totalReplicatedDataSize_ fields for
> tracking the total pending deletion size for FSO directories in OM.
> * {_}totalReplicatedDataSize{_}: Total size of directories pending deletion
> with expected replicated size.
> * {_}totalUnreplicatedDataSize{_}: Total raw size of directories pending
> deletion without replicated size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
