Nenad,
thank your answer!
1) this is only Haproxy server active (active/passive config exists,
but using carp on OpenBSD).
2) As I understand with nbcproc 4 I can't get stats working correctly ...
however at the moment I see that for https frontend I have :
Current connection rate: 58/s
Current session rate: 53/s
Current request rate: 124/s
For http frontend:
Current connection rate: 240/s
Current session rate: 240/s
Current request rate: 542/s
3) current top output (total in/out for HTTP/HTTPs traffic on external
interfaces is avg 300 Mbps and this is only Haproxy traffic):
load averages: 4.02, 3.92, 3.88
router2 19:28:18
32 processes: 1 running, 27 idle, 4 on processor
CPU0 states: 12.6% user, 0.0% nice, 11.2% system, 60.9% interrupt, 15.4% idle
CPU1 states: 25.2% user, 0.0% nice, 47.0% system, 0.2% interrupt, 27.6% idle
CPU2 states: 25.1% user, 0.0% nice, 43.3% system, 0.6% interrupt, 30.9% idle
CPU3 states: 21.6% user, 0.0% nice, 48.2% system, 0.2% interrupt, 30.0% idle
Memory: Real: 1017M/1709M act/tot Free: 14G Cache: 111M Swap: 0K/16G
PID USERNAME PRI NICE SIZE RES STATE WAIT TIME CPU COMMAND
24285 _haproxy 64 0 302M 154M onproc - 38:00 80.13% haproxy
26781 _haproxy 2 0 295M 147M run - 33:40 77.98% haproxy
26267 _haproxy 64 0 297M 149M onproc - 35:32 76.86% haproxy
23731 _haproxy 64 0 291M 143M onproc - 31:16 75.98% haproxy
On Sun, Apr 5, 2015 at 5:15 PM, Nenad Merdanovic <[email protected]> wrote:
> Evgeniy,
>
> On 4/5/2015 4:47 PM, Evgeniy Sudyr wrote:
>>
>> Lukas, thank you for pointing to possible keep-alive issues, I've
>> tested it before, but did it again just to make one more check!
>>
>> I've increased keep alives timeout to 10se and removed
>> http-server-close, restarted haproxy :)
>>
>>
>> Changes I've noted - haproxy reduced from AVG 78% to AVG 75% per each CPU
>> core.
>>
>> In top I see avg load is 4.10, before restart it was 4.20
>>
>> Average total bandwidth in frontend interfaces is 250 Mbps and of
>> course number of ESTABLISHED and all connections is much higer now
>> (which is OK, there are plenty of RAM there on this server):
>>
>> [root@router2 ~]# lsof -ni | grep haproxy | grep ESTABLISHED | grep
>> xxx.xxx.xxx.xxx | wc -l
>> 6568
>> [root@router2 ~]# lsof -ni | grep haproxy | grep xxx.xxx.xxx.xxx | wc -l
>> 6460
>
>
> Can you please send us the per-core usage (%usr, %sys, %si, ... from top for
> example) of your HAproxy box? How many RPS are you currently doing on the
> SSL frontend? Is this your only HAproxy server handling requests, as having
> something like ECMP towards multiple HAproxy servers would destory the point
> of SSL session cache.
>
>
>>
>> Interesting that interrupts % on CPU0 almost the same at 60%.
>>
>> Not that much CPU load decrease after changing keep alives, looks it's
>> something else.
>>
>>
>> On Sun, Apr 5, 2015 at 2:07 PM, Lukas Tribus <[email protected]> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> haproxy is used for http and https load balancing with TLS termination
>>>> on haproxy side.
>>>>
>>>> I'm using openbsd -stable on this box. I got CPU saturated with
>>>> 250Mbps traffic in/out summary on frontend NICs and 3000 ESTABLISHED
>>>> connections on frontent interface to haproxy.
>>>
>>>
>>>
>>> Remove:
>>> option http-server-close
>>> timeout http-keep-alive 1s
>>>
>>>
>>> and replace them with:
>>> option http-keep-alive
>>> option prefer-last-server
>>> timeout http-keep-alive 10s
>>>
>>>
>>>
>>> This will enable keep-alive mode with 10 seconds timeout, that should
>>> decrease the CPU load by an order of magnitude.
>>>
>>> The problem with this SSL/TLS terminating setups is the cost involved
>>> in the SSL/TLS handshake (the actual throughput doesn't really matter).
>>>
>>> Also, I suggest to remove the "no-tls-tickets" option, so that your
>>> clients
>>> can use both SSL sessions and TLS tickets to resume a SSL/TLS session
>>> without starting a full handshake.
>>>
>>>
>>>
>>> Lukas
>>>
>
> Regards,
> Nenad
--
--
With regards,
Eugene Sudyr
