Hi Aleks,

On Fri, Nov 06, 2026 at 11:36:28AM +0100, Aleksandar Lazic wrote:
> Hi.
> 
> While testing a latency aware variant of "balance random" (an RFC will
> follow), I noticed that the comparison of the draws is skipped more
> often than expected. random(<draws>) draws the servers on the
> consistent hash ring with replacement, so a draw can return the best
> server found so far. There is nothing to compare then, and the draw is
> lost. With 4 servers and 2 draws, this happens for one choice out of
> four, which then behaves like random(1).
> 
> The first patch draws again in this case, at most <draws> extra times
> so that a farm with a single usable server does not loop. It is a few
> lines in get_server_rnd(), plus one sentence in the documentation.
> 
> Measured locally with ideal servers (fixed delays, 30000 requests after
> a warm-up), share of the requests sent to the slow servers:
> 
> farm random(2) before -> after
> 4 servers, 1 at 50ms 8.73% -> 5.16%
> 8 servers, 2 at 50ms 7.5% -> 6.80%

Interesting. However I'm not fond of looping on extra draws, and we
don't need to because the consistent hash function already takes as
an argument the server to avoid, so we can use it to pass the previous
preferred server, which gives an even better distribution because
instead of having 25% chance of hitting the same server on each draw,
now we're down to 0% so we don't need to increase the count at all.

I'm testing with this config, having 1 server at 50ms and N-1 servers
at 5ms:

  defaults
      mode http
      timeout client 5s
      timeout server 5s
      timeout connect 5s

  frontend stats
      bind :8181
      stats uri /

  listen f
      bind :8001
      balance random(2)
      server s0 127.0.0.1:8003 # slow
      server s1 127.0.0.1:8002
      server s2 127.0.0.1:8002
      server s3 127.0.0.1:8002
      #server s4 127.0.0.1:8002
      #server s5 127.0.0.1:8002
      #server s6 127.0.0.1:8002
      #server s7 127.0.0.1:8002

  frontend fast
       bind :8002
       http-request pause 5ms
       http-request return status 200

  frontend slow
       bind :8003
       http-request pause 50ms
       http-request return status 200

Only the first server is slow (for ease of testing). In master I'm
getting this distribution for 100k req, 100 concurrent connections,
on 4 servers (server, max concurrent, total):

  srv  max  total
   s0   63   8525
   s1   27  28296
   s2   28  31049
   s3   29  32130

  Request rate: 11.1k

For 8 servers:

  srv  max  total
   s0   29   2248
   s1   17  13874
   s2   17  13869
   s3   18  13996
   s4   18  14316
   s5   17  14325
   s6   16  13758
   s7   16  13614

  Request rate: 15.8k

Now with this patch applied:

  --- a/src/backend.c
  +++ b/src/backend.c
  @@ -584,7 +584,7 @@ struct server *get_server_rnd(struct stream *s, const 
struct server *avoid)
          do {
                  prev = curr;
                  hash = statistical_prng();
  -               curr = chash_get_server_hash(px, hash, avoid);
  +               curr = chash_get_server_hash(px, hash, avoid ? avoid : prev);
                  if (!curr)
                          break;
 
4 servers:

  srv  max  total
   s0   29   3572
   s1   28  31380
   s2   29  32513
   s3   29  32535

  Request rate: 14.6k (+33%)

8 servers:

  srv  max  total
   s0   17   1669
   s1   15  14007
   s2   16  14010
   s3   15  14059
   s4   16  14205
   s5   16  14215
   s6   16  13968
   s7   15  13867

  Request rate: 16.6k (+5%)

The gains are clear, the ratios follow the response times, and the
distribution is also much smoother. I remember that when I wrote
that old article about P2C I mentioned that more draws were cleaner
(3+ is better) and allowed to avoid such cases as ending on the same
server, but by then that didn't flow into the code though.

> The second patch documents the other source of unevenness I found:
> with the default weight of 1, each server only gets 16 points on the
> ring, so the shares of the servers vary between about 2% and 4% with
> 32 servers. The same higher weight for all servers, such as 100, makes
> them almost even. I only documented it, as changing the number of
> points would change the distribution of the existing setups.
> 
> I would be glad about any feedback, also if you would prefer to keep
> the current behavior of random for some reason I did not think of.

I'm going to merge my smaller change, crediting you for the suggestion,
and your doc patch.

Thank you!
Willy


Reply via email to