Re: Weighted Backend's

[Bruno Henc]

Hello James,

The Balance random should do exactly what you want in HAProxy 1.9 :

Check out 
https://www.haproxy.com/blog/haproxy-1-9-has-arrived/#random-load-balancing-algorithm


"We’ve added a new random load-balancing algorithm. When used, a random 
number will be chosen as the key for the consistent hashing function. In 
this mode, server weights are respected. Dynamic weight changes take 
effect immediately, as do new server additions."


So to get the desired effect, you could use balance random without 
server weights: this should yield  an uniform distribution.


However, the socket approach is probably the best approach if you really 
want to have the 503-response-on-cluster-failure scenario.


The reason why the socket approach is also probably the best bet in 
production environments is that when you are scaling SSL termination 
across multiple haproxy processes it is often the case that you will use 
unix sockets anyway to distribute the load, and you can cleverly reuse 
those sockets to do what you want.


So you would have haproxy in tcp mode binding to ports 80 and 443, 
creating multiple sockets which different processes would bind to to 
handle TLS (and handle http load balancing). By cleverly adjusting the 
weights you could reuse these sockets for A/B testing purposes. Also, 
this allows you to test different TLS ciphers as well (each process can 
define their own TLS ciphers).


I should also point out that you can test different haproxy versions if 
you split the haproxy version used for the level 4 load balancing and 
the haproxy version used for level 7 load balancing. However, that might 
be overkill in your setup.


Considering this approach with L4 load balancing -> sockets -> L7 load 
balancing works quite nicely at scale, I would recommend it, though 
there are a few pitfalls associated with this. Heavy testing of the 
HAProxy configuration in staging is recommended or chatting with our 
enterprise support to avoid nasty surprises.


Please note that you should consider setting send-proxy-v2 and 
accept-proxy respectively to preserve source IP information when using 
sockets. Also, tcp-request inspect-delay is your friend (multiples of 3s 
are recommended to handle tcp re-transmission + 1 e.g. 4s, 5s etc.).


As for the latency/performance impact of sockets, it would be nice to 
measure the overhead and see if something can be optimized, but again, 
depending on the scale you're working at, it might be negligible 
compared to the elephant in the room (TLS CPU overhead, especially with 
RSA).


Also, with sockets, you can disable routing traffic to a backend simply 
by toggling the state of the socket, which is pretty neat.


The crux of the matter is that use_backend doesn't allow you to simply 
split the traffic in half in an obvious way (e.g. by simply specifying a 
keyword), instead, you have to define an ACL that would to that. You 
could do that by setting a special header or a cookie in your 
application code which would allow you to construct an ACL rule to split 
the traffic on.


An useful feature would be to modify the use_backend syntax to allow for 
easier splitting across multiple backends. It might be possible that Lua 
can achieve this. Can't make any promises, but I will investigate 
further into this if I have some spare time.


Another approach would be to add a feature that automatically rebalances 
all weights if a server is added/removed (so you would just use one big 
backend): However, this is also achievable by appropriately wrapping the 
HAProxy API. Might be less painful than the alternatives.


Another approach if you are working with multiple haproxy instances 
would be to deploy different haproxy configurations to different servers 
(one haproxy server exclusively servers A traffic, another one B traffic).


To conclude, there are multiple approaches that might yield good results 
with HAProxy, and some good features for better testing could be 
developed, but one should consider if this should be done by HAProxy at all.


Your application might be able to make the decision instead in an 
environment you are more comfortable with: the programming language used 
by your team.


If you do A/B testing in your application, you can use the entire server 
fleet

instead of sacrificing 50% of your servers as guinea pigs.

What happens when e.g. the entire B backend starts serving 500 responses?
The load on your A backend will double. Are you sure you have the 
capacity for that?


Contrast that to implementing the change via a feature flag: if anything 
goes wrong with "B" you change one value in e.g. your database, and the 
entire server fleet switches to using "A".


Without knowing more about the environment you are performing A/B 
testing (production, development, staging), the scale you are working 
at, and which part of the stack you are A/B testing, it is difficult to 
come up with a recommendation.



Let me know if you have any questions.

Best 

Re: Weighted Backend's

[James Root]

Hey Aleks,

Thank you for the reply, I should have included my version. I am currently
using HAProxy 1.8, but moving up a version is a possibility. I understand
what your example is doing, but it has the same issue my original example
has I think, that I have to have one unix socket per cluster. In my case,
"cluster" is just a small collection of servers with the same service, but
there could be dozens of these clusters.

In our setup, an inbound request gets routed to the correct backend based
on the host header (in my original example, this backend would be
"haproxy-test"). But then I effectively want to A/B test between the two
clusters that can serve this backend. I could put ever server in this one
backend, with the proper weights, but that isn't exactly what I am looking
for. Ideally, I would like if one cluster goes out completely that 503s get
returned for any requests that would normally get round robined to that
cluster. The only way I could find to actually enforce weighting between
two clusters was to forward the request through a socket to a new
"frontend" (functionally this is the same as running a proxy instance per
cluster). This seems to work, but I am looking for a way to do it without
opening up a large amount of unix sockets.


On Wed, Feb 6, 2019 at 11:43 AM Aleksandar Lazic  wrote:

> Hi James.
>
> Am 06.02.2019 um 16:16 schrieb James Root:
> > Hi All,
> >
> > I am doing some research and have not really found a great way to
> configure
> > HAProxy to get the desired results. The problem I face is that I a
> service
> > backed by two separate collections of servers. I would like to split
> traffic
> > between these two clusters (either using percentages or weights).
> Normally, I
> > would configure a single backend and calculate my weights to get the
> desired
> > effect. However, for my use case, the list of servers can be update
> dynamically
> > through the API. To maintain correct weighting, I would then have to
> > re-calculate the weights of every entry to maintain a correct balance.
> >
> > An alternative I found was to do the following in my configuration file:
> >
> > backend haproxy-test
> > balance roundrobin
> > server cluster1 u...@cluster1.sock weight 90
> > server cluster2 u...@cluster2.sock weight 10
> >
> > listen cluster1
> > bind u...@cluster1.sock
> > balance roundrobin
> > server s1 127.0.0.1:8081 
> >
> > listen cluster2
> > bind u...@cluster2.sock
> > balance roundrobin
> > server s1 127.0.0.1:8082 
> > server s2 127.0.0.1:8083 
> >
> > This works, but is a bit nasty because it has to take another round trip
> through
> > the kernel. Ideally, there would be a way to accomplish this without
> having to
> > open unix sockets, but I couldn't find any examples or any leads in the
> haproxy
> > docs.
> >
> > I was wondering if anyone on this list had any ideas to accomplish this
> without
> > using extra unix sockets? Or an entirely different way to get the same
> effect?
>
> Well as we don't know which version of HAProxy do you use I will suggest
> you a
> solution based on 1.9.
>
> I would try to use the set-priority-* feature
>
>
> https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#4.2-http-request%20set-priority-class
>
> https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#4.2-http-request%20set-priority-offset
>
>
> https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.2-prio_class
>
> https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.2-prio_offset
>
> https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.3-src
>
> I would try the following, untested but I think you get the idea.
>
> frontend clusters
>
>   bind u...@cluster1.sock
>   bind u...@cluster2.sock
>
>   balance roundrobin
>
>   # I'm not sure if src works with unix sockets like this
>   # maybe you need to remove the unix@ part.
>   acl src-cl1 src u...@cluster1.sock
>   acl src-cl2 src u...@cluster2.sock
>
>   http-request set-priority-class -10s if src-cl1
>   http-request set-priority-class +10s if src-cl2
>
> #  http-request set-priority-offset 5s if LOGO
> #  http-request set-priority-offset 5s if LOGO
>
>   use_backend cluster1 if priority-class < 5
>   use_backend cluster2 if priority-class > 5
>
>
> backend cluster1
> server s1 127.0.0.1:8081
>
> backend cluster2
> server s1 127.0.0.1:8082
> server s2 127.0.0.1:8083
>
> There are a lot of fetching functions so maybe you find a better solution
> with
> another fetch function as I don't know your application.
>
> https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7
>
> In case you haven't seen it there is also a management interface for
> haproxy.
>
> https://cbonte.github.io/haproxy-dconv/1.9/management.html#9.3
> https://www.haproxy.com/blog/dynamic-configuration-haproxy-runtime-api/
>
> > Thanks,
> > James Root
>
> Regards
> Aleks
>

Re: Weighted Backend's

[Aleksandar Lazic]

Hi James.

Am 06.02.2019 um 16:16 schrieb James Root:
> Hi All,
> 
> I am doing some research and have not really found a great way to configure
> HAProxy to get the desired results. The problem I face is that I a service
> backed by two separate collections of servers. I would like to split traffic
> between these two clusters (either using percentages or weights). Normally, I
> would configure a single backend and calculate my weights to get the desired
> effect. However, for my use case, the list of servers can be update 
> dynamically
> through the API. To maintain correct weighting, I would then have to
> re-calculate the weights of every entry to maintain a correct balance.
>
> An alternative I found was to do the following in my configuration file:
>
> backend haproxy-test
> balance roundrobin
> server cluster1 u...@cluster1.sock weight 90
> server cluster2 u...@cluster2.sock weight 10
> 
> listen cluster1
>     bind u...@cluster1.sock
>     balance roundrobin
>     server s1 127.0.0.1:8081 
> 
> listen cluster2
>     bind u...@cluster2.sock
>     balance roundrobin
>     server s1 127.0.0.1:8082 
>     server s2 127.0.0.1:8083 
> 
> This works, but is a bit nasty because it has to take another round trip 
> through
> the kernel. Ideally, there would be a way to accomplish this without having to
> open unix sockets, but I couldn't find any examples or any leads in the 
> haproxy
> docs.
> 
> I was wondering if anyone on this list had any ideas to accomplish this 
> without
> using extra unix sockets? Or an entirely different way to get the same effect?

Well as we don't know which version of HAProxy do you use I will suggest you a
solution based on 1.9.

I would try to use the set-priority-* feature

https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#4.2-http-request%20set-priority-class
https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#4.2-http-request%20set-priority-offset

https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.2-prio_class
https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.2-prio_offset

https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.3-src

I would try the following, untested but I think you get the idea.

frontend clusters

  bind u...@cluster1.sock
  bind u...@cluster2.sock

  balance roundrobin

  # I'm not sure if src works with unix sockets like this
  # maybe you need to remove the unix@ part.
  acl src-cl1 src u...@cluster1.sock
  acl src-cl2 src u...@cluster2.sock

  http-request set-priority-class -10s if src-cl1
  http-request set-priority-class +10s if src-cl2

#  http-request set-priority-offset 5s if LOGO
#  http-request set-priority-offset 5s if LOGO

  use_backend cluster1 if priority-class < 5
  use_backend cluster2 if priority-class > 5


backend cluster1
server s1 127.0.0.1:8081

backend cluster2
server s1 127.0.0.1:8082
server s2 127.0.0.1:8083

There are a lot of fetching functions so maybe you find a better solution with
another fetch function as I don't know your application.

https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7

In case you haven't seen it there is also a management interface for haproxy.

https://cbonte.github.io/haproxy-dconv/1.9/management.html#9.3
https://www.haproxy.com/blog/dynamic-configuration-haproxy-runtime-api/

> Thanks,
> James Root

Regards
Aleks