wyxxxcat opened a new pull request, #70810:
URL: https://github.com/apache/doris/pull/70810

   ### Background
   
   In Doris Cloud mode, data recycling periodically deletes unused rowsets and 
data files from object storage via batch or single deletion requests. 
Currently, request-level latency and QPS are tracked by 
`s3_delete_objects_latency` and `s3_delete_object_latency`.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: None
   PR Type: Enhancement
   
   Problem Summary:
   `s3_delete_objects_latency` only records the request-level latency and QPS 
of `DeleteObjects` calls. Because each batch delete request can contain up to 
1000 objects, operators cannot know how many objects were actually deleted or 
monitor the true object deletion throughput. This PR adds a new counter metric 
`s3_deleted_objects_total` to track the actual number of successfully deleted 
objects in both batch and single delete operations.
   
   ### Details
   
   - **Before and after:**
     - Before: Only request-level QPS (`s3_delete_objects_qps`) was available, 
which does not reflect the number of individual objects deleted per batch 
request.
     - After: `s3_deleted_objects_total` counts each successfully deleted 
object (accounting for partial failures in batch deletes and non-existent 
single delete failures), allowing Prometheus/Grafana to calculate actual object 
deletion QPS using `rate(s3_deleted_objects_total[1m])`.
   - **Configs and defaults:** None.
   - **Impact on existing users:** None. Existing metrics are unchanged; users 
get a new counter metric `s3_deleted_objects_total`.
   
   ### Testing
   
   Added unit tests in `cloud/test/s3_accessor_mock_test.cpp`:
   - `counts_successful_keys_in_each_delete_batch`: Verifies that 
`s3_deleted_objects_total` increments by the exact count of successful objects 
across batches, even when partial batch errors occur.
   - `failed_delete_request_does_not_increase_deleted_objects`: Verifies that 
request-level failure (e.g., HTTP 403) does not increase the counter.
   - `counts_only_successful_single_deletes`: Verifies that single object 
deletes increment by 1 on success and do not increment on NOT_FOUND or errors.
   
   ### Release note
   
   Added `s3_deleted_objects_total` metric to track the actual count of deleted 
objects in object storage.
   
   ### Check List (For Author)
   
   - Behavior changed: No
   - Does this need documentation: No
   - Pre-submission status: Ready


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to