priyeshkaratha opened a new pull request, #9326:
URL: https://github.com/apache/ozone/pull/9326
## What changes were proposed in this pull request?
A new REST API endpoint `/storagedistribution` has been introduced to
provide a summarized view of storage distribution across the cluster. This
endpoint is designed to consolidate various storage-related insights,
particularly focusing on used capacity, pending deletions, and overall data
distribution across Ozone components.
### Key Functionalities
1. **Endpoint Name:** `/storagedistribution`
* Exposes consolidated metrics related to the usage and distribution of
storage.
2. **Data Sources Used:**
* **OMDBInsightEndpoint:**
* Retrieves OM-level data such as bucket-level usage, namespace size,
and used capacity
* **SCMClient:**
* Fetches pending deletion metrics from SCM.
* This includes block-level metadata such as size and replicated size
info for accurate estimation of reclaimable space.
* **Datanode Storage Reports:**
* Gathers current storage usage statistics from individual datanodes.
* Helps in determining used vs available capacity and contributes to
classifying data distribution based on node-wise storage usage.
Here the pending deletion bytes are calculated and updated in the
BlockDeletionService metrics, which are then exposed via JMX. This allows for
direct access to these metrics by monitoring tools. The primary disadvantage of
this approach is the introduction of a dependency on JMX. While this might be a
minor concern in some deployments, it's worth noting the additional dependency.
This method avoids the unnecessary data transfer to SCM and the associated
processing overhead in DN while iterating and calculating pending deletion from
the container data list that is present with the Storage Report approach.
## What is the link to the Apache JIRA
HDDS-13465
## How was this patch tested?
Currently tested by unit test cases by mocking SCM and DN data. Also tested
manually using postman.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
