rajvarun77 commented on issue #3340: URL: https://github.com/apache/brpc/issues/3340#issuecomment-4743811054
## Benchmark: tail latency vs `rr`/`la` under slow backends I ran a tail-latency comparison using brpc's own load tool (`tools/rpc_press`, which reports p50/p99/p999 from `bvar::LatencyRecorder`) — no custom benchmark logic. Setup: 4 echo backends; a "slow" backend injects +5 ms per request (`bthread_usleep`, latency only, no CPU theft); fixed offered load `qps=4000`, `duration=15s`, `thread_num=50`, `list://` naming. This is the realistic "a few sick hosts" failure mode a latency-aware LB is meant to route around. Numbers are averaged over 3 reps. ### p99 / p999 latency (µs, lower is better) | scenario | `la` | `p2c` (best-of-2) | `p2c:choices=4` | `rr` | |---|---|---|---|---| | all healthy | 352 / 574 | 387 / 539 | 539 / 2921 | ~410 / 2821 | | **1 slow of 4** | 327 / 406 | 337 / 396 | 334 / 398 | 6474 / 9447 | | **2 slow of 4** | 303 / 360 | 7676 / 7736 | **307 / 385** | 7696 / 8149 | ### Findings 1. **No regression when healthy** — p2c ≈ `la` ≈ `rr` at the tail. 2. **One slow backend: p2c ≈ `la`, ~19× better p99 than `rr`** (337 µs vs 6474 µs). The slow node's peak-EWMA + in-flight count raise its score, so it loses nearly every comparison and drains within one observation; `rr` keeps sending it 25 % of traffic. 3. **Two slow backends: best-of-2 hits a sampling floor** — two random draws are both slow with probability `C(2,2)/C(4,2)=1/6`, so ~16 % of requests are forced onto a slow node. The configurable **`p2c:choices=4`** (best-of-N when N≤choices) evaluates all backends and recovers fully — **307 µs p99, matching `la`'s 303 µs** (~25× better than best-of-2). 4. **`choices` is a real tradeoff, not a free win** — at `choices=4` with a healthy fleet, p999 rises (2921 vs 539 µs) due to herding when all backends score alike. Hence it defaults to 2 and is meant to be raised only under heavy simultaneous degradation. ### Caveats Single-machine (loopback, client + servers share cores), so absolute latencies are not portable — the valid signal is the **relative** delta between policies under identical conditions. All runs were validated to have `success == sent` with zero errors before being included. Net: p2c gives `la`-class tail behavior with O(1) selection in the common case, plus a `choices` knob for the heavy-degradation regime where the global `la` tree would otherwise win. Implementation + the full reproducible harness (commands, raw CSV, plots) are ready; happy to open a PR if there's interest. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
