priyeshkaratha opened a new pull request, #9413: URL: https://github.com/apache/ozone/pull/9413
## Please describe your PR in detail: ### Overview This pull request introduces a new REST API endpoint `/pendingDeletion` in the Recon service to provide comprehensive visibility into pending block deletions across different components of the Ozone system. The endpoint allows administrators to monitor the status of pending deletions from three key components: 1. **DataNodes (DN)**: Pending blocks marked for deletion on individual datanodes 2. **Storage Container Manager (SCM)**: Aggregated deleted block transaction summary 3. **Ozone Manager (OM)**: Pending deletion information from the object store manager ### Changes Implemented #### 1. New REST Endpoint: `PendingDeletionEndpoint` - Routes to `/pendingDeletion` with support for a required `component` query parameter - Supports three component options: `dn`, `scm`, `om` - Implements asynchronous metric collection for DataNode metrics with proper status handling - Returns appropriate HTTP status codes (202 Accepted for ongoing tasks, 200 OK for completed results, 400 Bad Request for invalid parameters) #### 2. DataNode Metrics Service (`DataNodeMetricsService`) - A singleton service that manages the lifecycle of DataNode metric collection tasks - Uses a thread pool (10 threads) to parallelize metric collection from all available datanodes - Implements state management with four statuses: `NOT_STARTED`, `IN_PROGRESS`, `SUCCEEDED`, `FAILED` - Aggregates pending block deletion metrics across all datanodes - Tracks individual datanode metrics and total system-wide pending deletions - Includes timeout handling (30 seconds per datanode) to prevent indefinite waiting #### 3. DataNode Metrics Collection Task (`DataNodeMetricsCollectionTask`) - Callable task that fetches pending deletion metrics from individual datanodes via JMX endpoints - Queries the `BlockDeletingService` JMX bean to retrieve `TotalPendingBlockBytes` - Parses JSON response from the datanode's JMX metrics endpoint - Returns structured metrics in `DatanodePendingDeletionMetrics` format #### 4. Supporting Data Classes - `DatanodePendingDeletionMetrics`: Encapsulates per-datanode metrics (hostName, datanodeUuid, pendingBlockSize) - `DataNodeMetricsServiceResponse`: Builder-pattern response object containing collection status, total pending deletion size, and per-datanode metrics - `ScmPendingDeletion`: Contains SCM-level pending deletion information (totalBlockSize, totalBlockReplicatedSize, totalBlockCount) #### 5. Test Updates - Updated `TestStorageDistributionEndpoint.java` to support the new endpoint integration tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
