Hi Babtiste,
thanks a lot for your explanations! This totally makes sense, especially
from a HA and throughput perspective. My use case is different and I'll try
to explain how:
* 30 servers are generating emails
* 36 mail servers for sending out
* relatively cheap h/w: 2 bonded 1Gbit interfaces, SAS h/w raid
HAProxy is running locally on each of the email generating servers. HAProxy
is distributing emails over all available SMTP servers. Running HAProxy on
dedicated host(s) in between would create an SPOF and is not working from a
throughput perspective (30 times 2Gbit).
The email generating system is highly optimized and much quicker generating
emails than the MTAs are able to spool to disk. I've configured the maxconn
setting low to make sure the MTAs are not overwhelmed. The problem here is
that since I use 30 servers upstream, I effectively get 30 times 2 max.
concurrent sessions to a single MTA downstream. Increasing the value to 3
effectively means, 90. 4 means 120 concurrent connections and so on.
I've looked into solving this differently using SMTP status codes like 4xx
indicating resource exhaustion but that did not work well with the
smtp_health check or the upstream generating email system.
So why would I need a setting of weight 1 for a backend? Answer:
configuration changes. E.g. when changing settings at the MTA I would like
to run a small percentage of emails through this system and watch metrics
before moving to full load.
Maybe you have an idea how to solve this.
Thanks for your time!
Best regards,
Frank
2015-05-08 6:23 GMT-07:00 Baptiste :
> Hi Frank,
>
>
> My problem: The weight setting for a backend seems to be ignored when the
>> max concurrent session setting is reached. I was expecting the connection
>> to get queued for this backend but it seems to flip over to the host that
>> has connections available.
>>
>
> This is expected.
> The queue manager starts working as soon as at least one server as reached
> its maxconn. It has
> The weight is taken into account by the load-balancing algorithm, but
> since the queue manager has precedence over it.
> Hence as soon as a server has reached its maxconn, it is excluded from the
> load-balancing algorithm. Then the relative weight of the remaining server
> is applied.
>
>
>
>> I simplified my setup to 2 backend smtp servers, one with weight 100, the
>> other with weight 1. The max connection setting is set to 2. I'm opening
>> multiple SMTP connections simultaneously to this haproxy server. Attached
>> screenshot from haproxy stats shows that backend with weight 1 gets way too
>> many sessions.
>>
>> Increasing max concurrent sessions to 5 or more seem to prevent this
>> behavior, but I'm not totally sure about this.
>>
>
> The question is "Why do you need a maxconn to 2 on your servers?"
>
>
>
>> I would like to have only a small fraction (100:1) of requests go to the
>> backend with the lower weight and wonder how to do this correctly. It's
>> more important to me to have a defined distribution of connections going to
>> backends than answering requests as quickly as possible regardless of what
>> backend is used.
>>
>>
> the weight is the good way to go, simply increase your maxconn, unless
> there is a good reason for it to be as low as 2.
>
> Baptiste
>
>
>
