wyxxxcat opened a new pull request, #70810:
URL: https://github.com/apache/doris/pull/70810
### Background
In Doris Cloud mode, data recycling periodically deletes unused rowsets and
data files from object storage via batch or single deletion requests.
Currently, request-level latency and QPS are tracked by
`s3_delete_objects_latency` and `s3_delete_object_latency`.
### What problem does this PR solve?
Issue Number: None
Related PR: None
PR Type: Enhancement
Problem Summary:
`s3_delete_objects_latency` only records the request-level latency and QPS
of `DeleteObjects` calls. Because each batch delete request can contain up to
1000 objects, operators cannot know how many objects were actually deleted or
monitor the true object deletion throughput. This PR adds a new counter metric
`s3_deleted_objects_total` to track the actual number of successfully deleted
objects in both batch and single delete operations.
### Details
- **Before and after:**
- Before: Only request-level QPS (`s3_delete_objects_qps`) was available,
which does not reflect the number of individual objects deleted per batch
request.
- After: `s3_deleted_objects_total` counts each successfully deleted
object (accounting for partial failures in batch deletes and non-existent
single delete failures), allowing Prometheus/Grafana to calculate actual object
deletion QPS using `rate(s3_deleted_objects_total[1m])`.
- **Configs and defaults:** None.
- **Impact on existing users:** None. Existing metrics are unchanged; users
get a new counter metric `s3_deleted_objects_total`.
### Testing
Added unit tests in `cloud/test/s3_accessor_mock_test.cpp`:
- `counts_successful_keys_in_each_delete_batch`: Verifies that
`s3_deleted_objects_total` increments by the exact count of successful objects
across batches, even when partial batch errors occur.
- `failed_delete_request_does_not_increase_deleted_objects`: Verifies that
request-level failure (e.g., HTTP 403) does not increase the counter.
- `counts_only_successful_single_deletes`: Verifies that single object
deletes increment by 1 on success and do not increment on NOT_FOUND or errors.
### Release note
Added `s3_deleted_objects_total` metric to track the actual count of deleted
objects in object storage.
### Check List (For Author)
- Behavior changed: No
- Does this need documentation: No
- Pre-submission status: Ready
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]