[
https://issues.apache.org/jira/browse/HDDS-14985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDDS-14985:
-----------------------------------
Status: Patch Available (was: Open)
> [Docs] Explain "exclusive size" and "referenced size" of a snapshot
> -------------------------------------------------------------------
>
> Key: HDDS-14985
> URL: https://issues.apache.org/jira/browse/HDDS-14985
> Project: Apache Ozone
> Issue Type: Task
> Components: documentation, Snapshot
> Reporter: Wei-Chiu Chuang
> Assignee: Wei-Chiu Chuang
> Priority: Major
> Labels: pull-request-available
>
> It's not all that clear what "exclusive size" and "referenced size" mean.
> Elaborate it in the Ozone Snapshot overview doc:
> [https://ozone.apache.org/docs/administrator-guide/operations/snapshots/overview/]
>
> In the context of Apache Ozone, the exclusiveSize of a snapshot represents
> the amount of data that is held only by that specific snapshot and no other
> entity in the system.
> To understand it better, it helps to compare it with referencedSize:
> 1. Exclusive Size vs. Referenced Size
> * Referenced Size: This is the "Total View" size. It is the sum of all
> files and directories visible when you look into that snapshot. If your
> bucket has
> 100GB of data and you take a snapshot, the referencedSize is 100GB.
> * Exclusive Size: This is the "Unique" size. It is the data that exists in
> the system exclusively because this snapshot is keeping it alive. If you just
> took a snapshot and haven't deleted anything from the main bucket yet,
> the exclusiveSize is 0 bytes.
> 2. How exclusiveSize grows
> A snapshot's exclusive size increases when data is removed from the "newer"
> parts of the chain. Specifically, a file's size moves into a snapshot's
> exclusiveSize when:
> 1. The file is deleted or overwritten in the active bucket (AOS).
> 2. The file is not present in any newer snapshots in the same path chain.
> Example Scenario:
> 1. Time T1: You have File-A (1GB) in your bucket.
> 2. Time T2: You take Snapshot-1.
> * Snapshot-1 Referenced: 1GB
> * Snapshot-1 Exclusive: 0GB (because File-A still exists in the active
> bucket).
> 3. Time T3: You delete File-A from the active bucket.
> * Snapshot-1 Referenced: 1GB
> * Snapshot-1 Exclusive: 1GB (because Snapshot-1 is now the only thing
> in the system that "owns" File-A).
> 3. Why is it important?
> The exclusiveSize is the most critical metric for storage management
> because it tells you exactly how much space will be reclaimed if you delete
> that
> snapshot.
> When the SnapshotDeletingService (which you looked at earlier) runs, it
> identifies this exclusive data and marks it for deletion in the Datanodes. If
> a
> snapshot has an exclusiveSize of 50GB, deleting it will eventually free up
> 50GB of disk space.
> 4. Calculation in the Background
> Ozone calculates these sizes using a background service called the
> SnapshotSstFilteringService and the SnapshotDeletingService. They compare the
> "Deleted
> Table" of a snapshot with newer snapshots in the chain to determine if a
> block is truly exclusive or if it's still being "referenced" by a neighbor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
