Re: High performance HAProxy
Re-posting due to bad formatting (sorry folks) Willy, all, I am new to the list and let me use the chance and express my gratitude for this wonderful piece of software. Truly amazing. Thank you Willy, thank you all guys. With few folks here we had some learning and already are experiencing quite good results with HAProxy. Wanted to first of all share that during the tests we achieved up to 45,000 requests per second on SSL on a single 1G box (with same setup/hw below). isn't that amazing? :) Also wanted to ask for your opinion or advise on how we can possibly improve the setup further. It really feels like there is something more out there and we could tune up the setup further. Our use case is: - high request per second traffic (very high PPS/packet per second) - HTTPS - hundreds of thousands of requests per second - gigabytes of traffic /per second - currently handled by hardware LoadBalancers --> aim to replace hardware LoadBalancers with HAProxy What do we have currently in HAProxy: Rate: 26,000 HTTPS requests per second, per single HAProxy server CPU idle: 50% System avg load: 8 Software IRQs %: ~10% What would be great to have: - reduced system load - more idle CPU - ability to push more bandwidth or more requests per second - no Software IRQs (or less), possibly less context switches/interrupts Do you think it's possible to further improve current setup software/configuration wise? Lots of details below. Any help or advise is highly appreciated. Thanks, Eduard -- OS/HW Debian 7.8 CPU: 2xE5-2630 2.30GHz (24 threads) NIC: Intel i350-AM4 1 GbE Quad Port TRAFFIC FLOW: Traffic -> HAProxy Server Public Interface -> HAProxy Server Private Interface -> Backend Server(s) Private Interface and back BANDWIDTH: 38 MByte/s in, 40MByte/out (eth0 public, eth1 - private) --net/eth0net/eth1- recv send: recv send 24M 12M: 13M 28M CONFIGURATION and STATS sysctl: http://pastebin.com/raw.php?i=fiaKcA6P haproxy.cfg: http://pastebin.com/raw.php?i=WvsWKfHa /proc/interrupts: http://pastebin.com/raw.php?i=8xc3S0u5 mpstat 1 60: http://pastebin.com/raw.php?i=fD4R5wZR Some more stats using sar and dstat: http://pastebin.com/raw.php?i=x4WBnaGB TCP Stats (avg): Total: 9500 Established: 8150 Closed: 1000 Orphaned: 2 Timewait: 1000 Ports open: 5000 HA-Proxy version 1.5.12 OpenSSL: openssl-1.0.2a Compiled with: export LIBSSLBUILD=/tmp/libsslbuild OpenSSL: ./config --prefix=$LIBSSLBUILD no-shared no-ssl2 no-ssl3 -DOPENSSL_USE_IPV6=0 no-err enable-ec_nistp_64_gcc_128 zlib HAProxy: make TARGET=linux2628 CPU=native USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_TFO=1 ADDINC=-I$LIBSSLBUILD/include ADDLIB="-L$LIBSSLBUILD/lib -ldl"
High performance HAProxy
Willy, all, I am new to the list and let me use the chance and express my gratitude for this wonderful piece of software. Truly amazing. Thank you Willy, thank you all guys. With few folks here we had some learning and already are experiencing quite good results with HAProxy. Wanted to first of all share that during the tests we achieved up to 45,000 requests per second on SSL on a single 1G box (with same setup/hw below). isn't that amazing? :) Also wanted to ask for your opinion or advise on how we can possibly improve the setup further. It really feels like there is something more out there and we could tune up the setup further. Our use case is:- high request per second traffic (very high PPS/packet per second)- HTTPS- hundreds of thousands of requests per second- gigabytes of traffic /per second- currently handled by hardware LoadBalancers --> aim to replace hardware LoadBalancers with HAProxy What do we have currently in HAProxy:Rate: 26,000 HTTPS requests per second, per single HAProxy serverCPU idle: 50%System avg load: 8Software IRQs %: ~10% What would be great to have:- reduced system load- more idle CPU- ability to push more bandwidth or more requests per second- no Software IRQs (or less), possibly less context switches/interrupts Do you think it's possible to further improve current setup software/configuration wise? Lots of details below. Any help or advise is highly appreciated. Thanks,Eduard -- OS/HW Debian 7.8 CPU: 2xE5-2630 2.30GHz (24 threads) NIC: Intel i350-AM4 1 GbE Quad Port TRAFFIC FLOW: Traffic -> HAProxy Server Public Interface -> HAProxy Server Private Interface -> Backend Server(s) Private Interface and back BANDWIDTH: 38 MByte/s in, 40MByte/out (eth0 public, eth1 - private)--net/eth0net/eth1- recv send: recv send 24M 12M: 13M 28M CONFIGURATION and STATS sysctl: http://pastebin.com/raw.php?i=fiaKcA6P haproxy.cfg: http://pastebin.com/raw.php?i=WvsWKfHa /proc/interrupts: http://pastebin.com/raw.php?i=8xc3S0u5 mpstat 1 60: http://pastebin.com/raw.php?i=fD4R5wZR Some more stats using sar and dstat: http://pastebin.com/raw.php?i=x4WBnaGBTCP Stats (avg): Total: 9500 Established: 8150 Closed: 1000 Orphaned: 2 Timewait: 1000 Ports open: 5000 HA-Proxy version 1.5.12OpenSSL: openssl-1.0.2a Compiled with:export LIBSSLBUILD=/tmp/libsslbuildOpenSSL:./config --prefix=$LIBSSLBUILD no-shared no-ssl2 no-ssl3 -DOPENSSL_USE_IPV6=0 no-err enable-ec_nistp_64_gcc_128 zlib HAProxy:make TARGET=linux2628 CPU=native USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 USE_TFO=1 ADDINC=-I$LIBSSLBUILD/include ADDLIB="-L$LIBSSLBUILD/lib -ldl"
Connection refuse on client after configuring haproy
We have HAProxy running on GlusterFS and geting “connection refuse , port map failure “ any ideas ? Alex
Re: log SSL/TLS protocol version
Hello, On 6/9/2015 5:44 PM, Sylvain Faivre wrote: > Hello, > > We use Haproxy in front of HTTP servers, SSL termination is done on > HAproxy. > > Is there a way to have HAproxy log the SSL or TLS protocol version (TLS > 1.0 / 1.1 / 1.2) or specific cipher that was used for requests ? > Yes, you can use ssl_fc_protocol and ssl_fc_cipher, check the following: http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#7.3.4-ssl_fc_protocol http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#7.3.4-ssl_fc_cipher Just put them inside %[] in your log-format string. > I know this is negociated between each client and the HAproxy server, > but I would like to know which clients use outdated protocols. > > Thanks. > > Regards, Nenad
log SSL/TLS protocol version
Hello, We use Haproxy in front of HTTP servers, SSL termination is done on HAproxy. Is there a way to have HAproxy log the SSL or TLS protocol version (TLS 1.0 / 1.1 / 1.2) or specific cipher that was used for requests ? I know this is negociated between each client and the HAproxy server, but I would like to know which clients use outdated protocols. Thanks.
200W led high bay,industrial lighting, high bay light led
Hello, Hot sales outdoor 100w COB led floodlight LED HIGH BAY LIGHTING 30W led high bay 25$usd each pcs100W led high bay 45$usd each pcs150W led high bay only 58$usd each pcsLED FLOOD LIGHTING 10W only 3.3$usd each pcs50W only 12.5$usd each pcs80W only 22$usd each pcs We supply led lamp with high quality and competitive price. Hope to cooperate with you. Best Regards -- Kathy Wu Skype: kathystar11 JIN WANG Optoelectronics Co., Limited T: 0086 0755 33165048
2015 Latest professional police HD video camera
DearSiror=Madam, Hopethingsarewellwithyou. Bytheway,Wesup=ply oneof oursuperprofessionalpoliceHDcameraswhichse=llespeciallywellforyourreference.Everymonth ,weallexpor=tover5000pcstoUSA and Europe.,welcomeyourinquiries&nb=sp;,thanks. OEM,ODMarebothavailalbe. Welookforwardtohearingfrom=you. BestRegardsJenny =20ContactUS:Company Name:SmartShineTechnologyCo;Ltd.Address:2Building,Ton=gle IndustryPark ,NanshanRoad,Nanshan Industry,She=nzhen,China.ContacPerson;JennyYi=Tel:008613528871-2=67Fax:008675526187082Skype : polly=yi471Email: pollyyuehe@h=otmail.com= info@szsmartshine.comWebsite:www.szsmartshine=.com = =20
Re: [PATCH] remove codes for cleaning p->block_rules
On Tue, Jun 09, 2015 at 08:04:56PM +0800, Godbach wrote: > The patch is generated in 1.5 branch because I failed to update 1.6 > branch(maybe caused by my local network). As you said, both 1.6 and 1.5 > branches should apply this patch. Applied to 1.6 now, thanks! willy
Re: [PATCH] remove codes for cleaning p->block_rules
Hi Willy, On 2015/6/9 15:51, Willy Tarreau wrote: Yes, please feel free to do so, we'll backport it into 1.5 as well. BTW, I only checked this issue in 1.5 branch. It must affect 1.6 as well in my opinion. Attached is the patch. The patch is generated in 1.5 branch because I failed to update 1.6 branch(maybe caused by my local network). As you said, both 1.6 and 1.5 branches should apply this patch. -- Best Regards, Godbach >From f8fa9c908b5b817e1a5804584bc8433ab91f4767 Mon Sep 17 00:00:00 2001 From: Godbach Date: Tue, 9 Jun 2015 19:41:52 +0800 Subject: [PATCH] CLEANUP: deinit: remove codes for cleaning p->block_rules Since all rules listed in p->block_rules have been moved to the beginning of the http-request rules in check_config_validity(), there is no need to clean p->block_rules in deinit(). Signed-off-by: Godbach --- src/haproxy.c |6 -- 1 files changed, 0 insertions(+), 6 deletions(-) diff --git a/src/haproxy.c b/src/haproxy.c index 053..eac6f44 100644 --- a/src/haproxy.c +++ b/src/haproxy.c @@ -1020,12 +1020,6 @@ void deinit(void) free(cwl); } - list_for_each_entry_safe(cond, condb, &p->block_rules, list) { - LIST_DEL(&cond->list); - prune_acl_cond(cond); - free(cond); - } - list_for_each_entry_safe(cond, condb, &p->mon_fail_cond, list) { LIST_DEL(&cond->list); prune_acl_cond(cond); -- 1.7.7
開 發 《嘌》
??dfdc@@@ @?g?%%% 2015-06-09 16:32:12 2015-6-9
Re: [PATCH] remove codes for cleaning p->block_rules
Hi Godbach, On Mon, Jun 08, 2015 at 09:32:10PM +0800, Godbach wrote: > Hi Willy, > > Since all block rules has been move to the beginning of the http-request > rules in check_config_validity() by the the following codes: > > /* move any "block" rules at the beginning of the > http-request rules */ > if (!LIST_ISEMPTY(&curproxy->block_rules)) { > /* insert block_rules into http_req_rules at the > beginning */ > curproxy->block_rules.p->n= > curproxy->http_req_rules.n; > curproxy->http_req_rules.n->p = > curproxy->block_rules.p; > curproxy->block_rules.n->p= > &curproxy->http_req_rules; > curproxy->http_req_rules.n= > curproxy->block_rules.n; > LIST_INIT(&curproxy->block_rules); > } I didn't remember we did this :-) > As a result, there is no need to clean blocking rules in deinit() as below: > > list_for_each_entry_safe(cond, condb, &p->block_rules, list) > { > LIST_DEL(&cond->list); > prune_acl_cond(cond); > free(cond); > } Indeed! > In addition, there is also another issue. The type of the members listed > in block_rules has become *struct http_req_rule*, not *struct acl_cond* > in earlier versions, maybe there is also potential risk to clean > block_rules in deinit(). > > So in my opinion, just remove the codes will be OK as below: > > diff --git a/src/haproxy.c b/src/haproxy.c > index 053..eac6f44 100644 > --- a/src/haproxy.c > +++ b/src/haproxy.c > @@ -1020,12 +1020,6 @@ void deinit(void) > free(cwl); > } > > - list_for_each_entry_safe(cond, condb, &p->block_rules, list) > { > - LIST_DEL(&cond->list); > - prune_acl_cond(cond); > - free(cond); > - } > - That's OK for me. > list_for_each_entry_safe(cond, condb, &p->mon_fail_cond, > list) { > LIST_DEL(&cond->list); > prune_acl_cond(cond); > > > I can send a patch later if there is no problem. Yes, please feel free to do so, we'll backport it into 1.5 as well. > BTW, I only checked this issue in 1.5 branch. It must affect 1.6 as well in my opinion. Thanks, Willy