bobhan1 opened a new pull request, #66977:
URL: https://github.com/apache/doris/pull/66977
### What problem does this PR solve?
Issue Number: N/A
Related PR: #66969, #66940
Problem Summary: BE host-level MetaService RPC rate limiting currently
sleeps inside the token-bucket add path, so operators cannot validate per-RPC
QPS thresholds against production traffic before enforcing them. This PR adds
`enable_ms_rpc_host_level_rate_limit_dry_run`, enabled by default, to evaluate
the configured limits without delaying RPCs. Dry-run uses a separate token
bucket, records would-wait latency in the existing
`host_level_ms_rpc_rate_limit_sleep` metrics, and emits debug logs for
would-throttle requests. It takes precedence when both dry-run and enforcement
are enabled.
The implementation keeps `TokenBucketRateLimiterHolder` unchanged.
`TokenBucketRateLimiter` only exposes token reservation separately from
sleeping, while the BE Cloud path owns a dedicated
`DryRunTokenBucketRateLimiter` with independent state.
### Release note
Add `enable_ms_rpc_host_level_rate_limit_dry_run` to observe
BE-to-MetaService host-level rate limiting without delaying RPCs. It is enabled
by default and takes precedence over enforcement when both switches are enabled.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- `./run-be-ut.sh --run
--filter=HostLevelMSRpcRateLimitersTest.*:HostLevelMSRpcRateLimitersConfigTest.*
-j100` (11 tests passed)
- `./build.sh --be -j100`
- `build-support/check-format.sh`
- `build-support/check-build-hygiene.sh`
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [ ] No.
- [x] Yes. Dry-run observation is enabled by default and never delays
MetaService RPCs.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
