Re: webdav
Wicked, thanks for your answer. Sander On 10.10.2013 00:03, Bryan Talbot wrote: I've used it in front of SVN running in apache httpd and proxy in http mode with ssl. works great. -Bryan On Wed, Oct 9, 2013 at 1:59 AM, Sander Klein roe...@roedie.nl wrote: Hey Baptiste, We want to use it in front of svn and git. We wont actually do any load balancing with it. We just want to use haproxy for a single entry point to the repositories. Greets, Sander On 09.10.2013 09:31, Baptiste wrote: Hi Sander, As long as webdav respect HTTP RFC, there won't be any issues at all. Which product are you targeting for your webdav deployment? Baptiste On Wed, Oct 9, 2013 at 8:57 AM, Sander Klein roe...@roedie.nl wrote: Hi, Is it possible to use webdav with haproxy while in http mode? Or dou I have to use tcp mode for that? Regards, Sander
L'investissement le plus rentable et le plus sure
Title: ehpad Ehpad : L' investissement immobilier avec 11 % de rduction d'impôt sans risque locatif Le dispositif ehpad, qui tire sa spcificit de la loi Bouvard et du rgime de la location meuble non professionnelle, est devenu encore plus incitatif ces dernières annes. C’est aujourd'hui « un parfait outil de dfiscalisation». Il a pour avantage de permettre aux investisseurs individuels de bnficier : D'une rduction d'impôt de 11 % jusqu'à 300 000 euros tale sur neuf ans ehpad De rcuprer le montant total de la TVA à 19,6 % investissement ehpad De profiter en outre d'une imposition favorable des loyers ehpad Avec une rentabilit assure aux alentours de 4,50 % à 5,50 % par an, l'investissement en Ehpad permet de bnficier d'un revenu à long terme. Les particuliers qui dsirent investir en un programme Ehpad ou de la defiscalisation expatris peuvent se constituer un revenu complmentaire pour leur retraite mais galement rduire leur taux d'imposition. De fait, cet investissement permet de bnficier d'une rduction d'impôt de 11% du prix d'acquisition. Ehpad propose aussi des programmes dedefiscalisation solidaire L'investissement EHPAD permet de bnficier de nombreux avantages fiscaux et financiers à condition de slectionner ses partenaires avec prcaution. C'est entre autres sur ce point que nous vous accompagnons. L'investissement immobilier avec11% de rduction d'impôt sans risque locatif = L'EHPADPour savoir si vous pouvez bnficier de cet investissement, contactez nous sans aucun engagement PS: la seule condition, est d'avoir un revenu minimum de 2 500 par foyer ! A tout de suite, Service EHPAD France
Load Balancer Replacement
Just reading some of your work online that discusses replacing load balancers with haproxy. It looks like a great solution. http://haproxy.1wt.eu/ Unfortunately I am simply not smart enough to weed through the architecture details to decide whether the tool will work in my environment. Perhaps you can provide some insight for me. My servers are VM's and they run Windows Server 2008 R2. I am currently using load balancers and they sometimes introduce latency that causes primary servers to failover to secondary servers. I'm wondering if haproxy or one of the other suggestions in the Other Solutions section would work for me? Any input appreciated. Thanks, Bill Shervey Work: 321 499-2544 Cell:321 576-2629 Business Resiliency Technology Support Corporate Business Resiliency Strategic Initiative Lockheed Martin - Enterprise Business Services [cid:image001.png@01CEC5AF.BF75F350] inline: image001.png
Delays from HAProxy
Hello, I am trying to configure an HAProxy And seem to be running into a problem where the HAProxy will spike. I have a high throughput server cluster, which will need to handle about 10-20k QPS. I believe that HAProxy should be able to handle that quite easily from what I have read. I am running HAProxy on Ubuntu 12.04LTS. I am running an a Xeon 1270, with 8GB of RAM(Which i believe is all overkill). I am doing a simple load balance between two or three servers(I have two server clusters, one with two servers, one with three). I am running 1.4.24. The reason I believe something is wrong with my haproxy is because I can refresh my webpage a few times, and every 4-5 times I will be get a spike around 800ms-3sec to resolve. This also happens when not hitting my backend server, and instead hitting the HAProxy stats page. None of my servers are anywhere close to using up their system resources(all below 20% in both memory and CPU). I don't see much in the syslog that seems wrong other than this message: Oct 10 16:48:28 haproxy rsyslogd-2177: imuxsock begins to drop message 3912 due to rate-limiting I believe that has to do with the logging not being able to keep up, rather than haproxy dropping packets. I was wondering if there were any suggestions for me to look into to try and fix my problem, or suggest how to configure a HAProxy correctly to be able to handle the load I need(preferably higher than the load I need, as this may increase over time). I am open to changing operating systems if need be as well. Here are my two config files: global log /dev/log local0 info log /dev/log local0 notice maxconn 5 user y group y #debug defaults log global modehttp option httplog option dontlognull option forwardfor retries 3 option redispatch option http-server-close maxconn 3 contimeout 1 clitimeout 5 srvtimeout 5 balance leastconn listen c_cluster 255.255.255.146:80 maxconn 3 server c1 10.101.13.74:80 maxconn 1 server c2 10.101.13.78:80 maxconn 1 server c3 10.101.13.82:80 maxconn 1 listen stats : mode http stats enable stats hide-version stats uri / and # this config needs haproxy-1.1.28 or haproxy-1.2.1 global log /dev/log local0 info log /dev/log local0 notice maxconn 2 user y group y #debug defaults log global modehttp option httplog option dontlognull option forwardfor retries 3 option redispatch option http-server-close maxconn 2 contimeout 1 clitimeout 5 srvtimeout 5 balance leastconn listen a_cluster 255.255.255.151:80 maxconn 2 server a1 10.101.13.68:80 maxconn 1 server a2 10.101.13.66:80 maxconn 1 listen stats : mode http stats enable stats hide-version stats uri / My sysctl.conf has the following changes: fs.file-max = 100 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_tw_recycle = 1 Thank you, Andy
Re: Delays from HAProxy
Hi Andy, Can't see anything wrong with your configuration. How about the server running haproxy? anything about that regarding cpu etc? I wouldn't be using net.ipv4.tcp_tw_recycle = 1 tw_recycle is a bit dodgy and can give some unwanted side-effects. tw_reuse = 1 should be sufficient. How does a tcpdump look? Since you get this issue on the stats get a dump on that. What setting are there on the box regarding SYN backlog? net.core.somaxconn net.ipv4.tcp_max_syn_backlog Best regards, pelle On Thu, Oct 10, 2013 at 7:16 PM, Andy M. ajm2...@gmail.com wrote: Hello, I am trying to configure an HAProxy And seem to be running into a problem where the HAProxy will spike. I have a high throughput server cluster, which will need to handle about 10-20k QPS. I believe that HAProxy should be able to handle that quite easily from what I have read. I am running HAProxy on Ubuntu 12.04LTS. I am running an a Xeon 1270, with 8GB of RAM(Which i believe is all overkill). I am doing a simple load balance between two or three servers(I have two server clusters, one with two servers, one with three). I am running 1.4.24. The reason I believe something is wrong with my haproxy is because I can refresh my webpage a few times, and every 4-5 times I will be get a spike around 800ms-3sec to resolve. This also happens when not hitting my backend server, and instead hitting the HAProxy stats page. None of my servers are anywhere close to using up their system resources(all below 20% in both memory and CPU). I don't see much in the syslog that seems wrong other than this message: Oct 10 16:48:28 haproxy rsyslogd-2177: imuxsock begins to drop message 3912 due to rate-limiting I believe that has to do with the logging not being able to keep up, rather than haproxy dropping packets. I was wondering if there were any suggestions for me to look into to try and fix my problem, or suggest how to configure a HAProxy correctly to be able to handle the load I need(preferably higher than the load I need, as this may increase over time). I am open to changing operating systems if need be as well. Here are my two config files: global log /dev/log local0 info log /dev/log local0 notice maxconn 5 user y group y #debug defaults log global modehttp option httplog option dontlognull option forwardfor retries 3 option redispatch option http-server-close maxconn 3 contimeout 1 clitimeout 5 srvtimeout 5 balance leastconn listen c_cluster 255.255.255.146:80 maxconn 3 server c1 10.101.13.74:80 maxconn 1 server c2 10.101.13.78:80 maxconn 1 server c3 10.101.13.82:80 maxconn 1 listen stats : mode http stats enable stats hide-version stats uri / and # this config needs haproxy-1.1.28 or haproxy-1.2.1 global log /dev/log local0 info log /dev/log local0 notice maxconn 2 user y group y #debug defaults log global modehttp option httplog option dontlognull option forwardfor retries 3 option redispatch option http-server-close maxconn 2 contimeout 1 clitimeout 5 srvtimeout 5 balance leastconn listen a_cluster 255.255.255.151:80 maxconn 2 server a1 10.101.13.68:80 maxconn 1 server a2 10.101.13.66:80 maxconn 1 listen stats : mode http stats enable stats hide-version stats uri / My sysctl.conf has the following changes: fs.file-max = 100 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_tw_recycle = 1 Thank you, Andy
Fwd: Delays from HAProxy
Thanks for the response pelle, The haproxy box is running at about 10-15% cpu. Looking at the TCP Dump, it seems the client is doing 1 or more TCP_Retransmissions of the incoming request when there is a delay. Anyone know why this would happen? Here is the requested information: root@haproxy:~# sysctl net.core.somaxconn net.core.somaxconn = 128 root@haproxy:~# sysctl net.ipv4.tcp_max_syn_backlog net.ipv4.tcp_max_syn_backlog = 2048 I removed the tcp_tw_recycle, but this did not change anything Thank you, Andy On Thu, Oct 10, 2013 at 1:58 PM, Pär Åslund psl...@gmail.com wrote: Hi Andy, Can't see anything wrong with your configuration. How about the server running haproxy? anything about that regarding cpu etc? I wouldn't be using net.ipv4.tcp_tw_recycle = 1 tw_recycle is a bit dodgy and can give some unwanted side-effects. tw_reuse = 1 should be sufficient. How does a tcpdump look? Since you get this issue on the stats get a dump on that. What setting are there on the box regarding SYN backlog? net.core.somaxconn net.ipv4.tcp_max_syn_backlog Best regards, pelle On Thu, Oct 10, 2013 at 7:16 PM, Andy M. ajm2...@gmail.com wrote: Hello, I am trying to configure an HAProxy And seem to be running into a problem where the HAProxy will spike. I have a high throughput server cluster, which will need to handle about 10-20k QPS. I believe that HAProxy should be able to handle that quite easily from what I have read. I am running HAProxy on Ubuntu 12.04LTS. I am running an a Xeon 1270, with 8GB of RAM(Which i believe is all overkill). I am doing a simple load balance between two or three servers(I have two server clusters, one with two servers, one with three). I am running 1.4.24. The reason I believe something is wrong with my haproxy is because I can refresh my webpage a few times, and every 4-5 times I will be get a spike around 800ms-3sec to resolve. This also happens when not hitting my backend server, and instead hitting the HAProxy stats page. None of my servers are anywhere close to using up their system resources(all below 20% in both memory and CPU). I don't see much in the syslog that seems wrong other than this message: Oct 10 16:48:28 haproxy rsyslogd-2177: imuxsock begins to drop message 3912 due to rate-limiting I believe that has to do with the logging not being able to keep up, rather than haproxy dropping packets. I was wondering if there were any suggestions for me to look into to try and fix my problem, or suggest how to configure a HAProxy correctly to be able to handle the load I need(preferably higher than the load I need, as this may increase over time). I am open to changing operating systems if need be as well. Here are my two config files: global log /dev/log local0 info log /dev/log local0 notice maxconn 5 user y group y #debug defaults log global modehttp option httplog option dontlognull option forwardfor retries 3 option redispatch option http-server-close maxconn 3 contimeout 1 clitimeout 5 srvtimeout 5 balance leastconn listen c_cluster 255.255.255.146:80 maxconn 3 server c1 10.101.13.74:80 maxconn 1 server c2 10.101.13.78:80 maxconn 1 server c3 10.101.13.82:80 maxconn 1 listen stats : mode http stats enable stats hide-version stats uri / and # this config needs haproxy-1.1.28 or haproxy-1.2.1 global log /dev/log local0 info log /dev/log local0 notice maxconn 2 user y group y #debug defaults log global modehttp option httplog option dontlognull option forwardfor retries 3 option redispatch option http-server-close maxconn 2 contimeout 1 clitimeout 5 srvtimeout 5 balance leastconn listen a_cluster 255.255.255.151:80 maxconn 2 server a1 10.101.13.68:80 maxconn 1 server a2 10.101.13.66:80 maxconn 1 listen stats : mode http stats enable stats hide-version stats uri / My sysctl.conf has the following changes: fs.file-max = 100 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_tw_recycle = 1 Thank you, Andy
Re: Delays from HAProxy
Thanks for the response pelle, The haproxy box is running at about 10-15% cpu. Looking at the TCP Dump, it seems the client is doing 1 or more TCP_Retransmissions of the incoming request when there is a delay. Anyone know why this would happen? Here is the requested information: root@haproxy:~# sysctl net.core.somaxconn net.core.somaxconn = 128 root@haproxy:~# sysctl net.ipv4.tcp_max_syn_backlog net.ipv4.tcp_max_syn_backlog = 2048 I removed the tcp_tw_recycle, but this did not change anything Thank you, Andy On Thu, Oct 10, 2013 at 3:44 PM, Andy M. ajm2...@gmail.com wrote: Thanks for the response pelle, The haproxy box is running at about 10-15% cpu. Looking at the TCP Dump, it seems the client is doing 1 or more TCP_Retransmissions of the incoming request when there is a delay. Anyone know why this would happen? Here is the requested information: root@haproxy:~# sysctl net.core.somaxconn net.core.somaxconn = 128 root@haproxy:~# sysctl net.ipv4.tcp_max_syn_backlog net.ipv4.tcp_max_syn_backlog = 2048 I removed the tcp_tw_recycle, but this did not change anything Thank you, Andy On Thu, Oct 10, 2013 at 1:58 PM, Pär Åslund psl...@gmail.com wrote: Hi Andy, Can't see anything wrong with your configuration. How about the server running haproxy? anything about that regarding cpu etc? I wouldn't be using net.ipv4.tcp_tw_recycle = 1 tw_recycle is a bit dodgy and can give some unwanted side-effects. tw_reuse = 1 should be sufficient. How does a tcpdump look? Since you get this issue on the stats get a dump on that. What setting are there on the box regarding SYN backlog? net.core.somaxconn net.ipv4.tcp_max_syn_backlog Best regards, pelle On Thu, Oct 10, 2013 at 7:16 PM, Andy M. ajm2...@gmail.com wrote: Hello, I am trying to configure an HAProxy And seem to be running into a problem where the HAProxy will spike. I have a high throughput server cluster, which will need to handle about 10-20k QPS. I believe that HAProxy should be able to handle that quite easily from what I have read. I am running HAProxy on Ubuntu 12.04LTS. I am running an a Xeon 1270, with 8GB of RAM(Which i believe is all overkill). I am doing a simple load balance between two or three servers(I have two server clusters, one with two servers, one with three). I am running 1.4.24. The reason I believe something is wrong with my haproxy is because I can refresh my webpage a few times, and every 4-5 times I will be get a spike around 800ms-3sec to resolve. This also happens when not hitting my backend server, and instead hitting the HAProxy stats page. None of my servers are anywhere close to using up their system resources(all below 20% in both memory and CPU). I don't see much in the syslog that seems wrong other than this message: Oct 10 16:48:28 haproxy rsyslogd-2177: imuxsock begins to drop message 3912 due to rate-limiting I believe that has to do with the logging not being able to keep up, rather than haproxy dropping packets. I was wondering if there were any suggestions for me to look into to try and fix my problem, or suggest how to configure a HAProxy correctly to be able to handle the load I need(preferably higher than the load I need, as this may increase over time). I am open to changing operating systems if need be as well. Here are my two config files: global log /dev/log local0 info log /dev/log local0 notice maxconn 5 user y group y #debug defaults log global modehttp option httplog option dontlognull option forwardfor retries 3 option redispatch option http-server-close maxconn 3 contimeout 1 clitimeout 5 srvtimeout 5 balance leastconn listen c_cluster 255.255.255.146:80 maxconn 3 server c1 10.101.13.74:80 maxconn 1 server c2 10.101.13.78:80 maxconn 1 server c3 10.101.13.82:80 maxconn 1 listen stats : mode http stats enable stats hide-version stats uri / and # this config needs haproxy-1.1.28 or haproxy-1.2.1 global log /dev/log local0 info log /dev/log local0 notice maxconn 2 user y group y #debug defaults log global modehttp option httplog option dontlognull option forwardfor retries 3 option redispatch option http-server-close maxconn 2 contimeout 1 clitimeout 5 srvtimeout 5 balance leastconn listen a_cluster 255.255.255.151:80 maxconn 2 server a1 10.101.13.68:80 maxconn 1 server a2 10.101.13.66:80 maxconn 1 listen stats : mode http stats enable
Need help with 1.5 crashing when browser refreshed
I am having an issue with the 1.5dev19 and the latest snapshot. It crashes sometimes when loading our site. It’s repeatable simply by refreshing the browser several times rapidly. I’ve tried using poll instead of kqueue and there was no change. Version 1.4 does not display the problem. I was hoping to get some help debugging this if possible. I’m not that familiar with the current C tools, but have some experience from years ago so if someone would point me to some info on setting this up to run in a debugger I’ll look into it further. Thanks, Kevin Process: haproxy [36011] Path:/usr/local/Cellar/haproxy/1.5-dev19/bin/haproxy Identifier: haproxy Version: 0 Code Type: X86-64 (Native) Parent Process: launchd [1] User ID: 0 Date/Time: 2013-10-10 10:53:42.811 -0500 OS Version: Mac OS X 10.8.5 (12F37) Report Version: 10 Crashed Thread: 0 Dispatch queue: com.apple.main-thread Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x VM Regions Near 0x: -- shared memory 7fe1b000-7fe1c000 [4K] r-x/r-x SM=SHM Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 haproxy 0x00010d6ffc8c session_accept + 95 1 haproxy 0x00010d6c3e73 listener_accept + 469 2 haproxy 0x00010d70da01 _do_poll + 1209 3 haproxy 0x00010d6bbd0c run_poll_loop + 51 4 haproxy 0x00010d6bc427 main + 1759 5 libdyld.dylib 0x7fff8e65f7e1 start + 1 Thread 0 crashed with X86 Thread State (64-bit): rax: 0x7fb1c406ce00 rbx: 0x rcx: 0x0001 rdx: 0x00010d73c018 rdi: 0x rsi: 0x rbp: 0x7fff52546bd0 rsp: 0x7fff52546b80 r8: 0x00010d73bc44 r9: 0x00010d73bc08 r10: 0x0040 r11: 0x7fb1c406ce00 r12: 0x0013 r13: 0x7fff52546bf0 r14: 0x7fb1c406ce00 r15: 0x7fb1c400c400 rip: 0x00010d6ffc8c rfl: 0x00010206 cr2: 0x Logical CPU: 6
Re: Need help with 1.5 crashing when browser refreshed
So after some more troubleshooting the problem seems to be related to PCRE. If I compile without it I don’t see the problem. In 1.4 there does not appear to be any issue using PCRE. Any ideas? - Kevin On Oct 10, 2013, at 4:23 PM, Kevin k...@gatorgraphics.com wrote: I am having an issue with the 1.5dev19 and the latest snapshot. It crashes sometimes when loading our site. It’s repeatable simply by refreshing the browser several times rapidly. I’ve tried using poll instead of kqueue and there was no change. Version 1.4 does not display the problem. I was hoping to get some help debugging this if possible. I’m not that familiar with the current C tools, but have some experience from years ago so if someone would point me to some info on setting this up to run in a debugger I’ll look into it further. Thanks, Kevin
Huge performance issues with Haproxy + SSL
I have been working on trouble shooting Haproxy 1.5 dev 19 with SSL for the last day or so on Cent OS 6.4 64bit. Lastest OpenSSL compiled 1.0.1e, recompiled haproxy with this make -s TARGET=linux2628 USE_EPOLL=1 USE_OPENSSL=1 ARCH=x86_64 clean all SSL cert wild card, plus godaddy intermediate and our key. Our current set of issues we are seeing: *Massive amounts of connection refused when running the test with ssl *Very High usage of CPU on this 8 core 32 gig box with 100 gig ssd and 1gb nic *Maybe 1/4 the amount of traffic we can push though, compaired to a non ssl test We are using Jmeter to load test and blazemeter to do up to 40k jmeter threads for a full hour. Here is a list of the errors that are spit back after the test is done Response codes response codecountresponse message 400 29 Bad request Non HTTP response code: javax.net.ssl.SSLPeerUnverifiedException 86069 Non HTTP response message: peer not authenticated Non HTTP response code: org.apache.http.conn.HttpHostConnectException 27229 Non HTTP response message: Connection to https://.com: refused Non HTTP response code: java.net.SocketException 88 Non HTTP response message: Connection reset 4122 Precondition Failed Non HTTP response code: org.apache.http.NoHttpResponseException270Non HTTP response message: The target server failed to respond So this is what we are facing and we are not haproxy experts and think we have taken it to the best of what we understand about haproxy config and settings. special note: we do not have a web site on the backend, its user server for an upcoming game we are working on so the stack is quite simple from haproxy - node.js -- db and back. Json data is posted to the user server and returned.
Re: Huge performance issues with Haproxy + SSL
Hi Steve, Can you send us your configuration (anonymised if required). We also need your sysctls (at least the one you've modified). Baptiste On Fri, Oct 11, 2013 at 4:43 AM, steve bloga...@gmail.com wrote: I have been working on trouble shooting Haproxy 1.5 dev 19 with SSL for the last day or so on Cent OS 6.4 64bit. Lastest OpenSSL compiled 1.0.1e, recompiled haproxy with this make -s TARGET=linux2628 USE_EPOLL=1 USE_OPENSSL=1 ARCH=x86_64 clean all SSL cert wild card, plus godaddy intermediate and our key. Our current set of issues we are seeing: *Massive amounts of connection refused when running the test with ssl *Very High usage of CPU on this 8 core 32 gig box with 100 gig ssd and 1gb nic *Maybe 1/4 the amount of traffic we can push though, compaired to a non ssl test We are using Jmeter to load test and blazemeter to do up to 40k jmeter threads for a full hour. Here is a list of the errors that are spit back after the test is done Response codes response codecountresponse message 400 29 Bad request Non HTTP response code: javax.net.ssl.SSLPeerUnverifiedException 86069 Non HTTP response message: peer not authenticated Non HTTP response code: org.apache.http.conn.HttpHostConnectException 27229 Non HTTP response message: Connection to https://.com: refused Non HTTP response code: java.net.SocketException 88 Non HTTP response message: Connection reset 4122 Precondition Failed Non HTTP response code: org.apache.http.NoHttpResponseException270Non HTTP response message: The target server failed to respond So this is what we are facing and we are not haproxy experts and think we have taken it to the best of what we understand about haproxy config and settings. special note: we do not have a web site on the backend, its user server for an upcoming game we are working on so the stack is quite simple from haproxy - node.js -- db and back. Json data is posted to the user server and returned.
Re: Delays from HAProxy
On Thu, Oct 10, 2013 at 03:57:37PM -0400, Andy M. wrote: Thanks for the response pelle, The haproxy box is running at about 10-15% cpu. Looking at the TCP Dump, it seems the client is doing 1 or more TCP_Retransmissions of the incoming request when there is a delay. Anyone know why this would happen? When you say the incoming request, you mean the SYN packet, right ? If so, that means the backlog is full. Here is the requested information: root@haproxy:~# sysctl net.core.somaxconn net.core.somaxconn = 128 This is far too low or 20k CPS. You fill it in 6 ms at 20k CPS. Try increasing it to 1024 to see how things go. root@haproxy:~# sysctl net.ipv4.tcp_max_syn_backlog net.ipv4.tcp_max_syn_backlog = 2048 In practice, I'm used to increase somaxconn to ~1 and max_syn_backlog to ~2, but it's a bit more tricky as you don't want somaxconn to be larger than haproxy's listen queue, otherwise the system will disable SYN cookies. In practice, 1024 should already be OK (about 51ms to fill it). I removed the tcp_tw_recycle, but this did not change anything OK. If things do not improve with the settings above, you'll have to check if you're running netfilter and to see if your contrack table is never full : # dmesg | grep -i full You also need to check if your network card or driver is not dropping packets : # netstat -i Regards, Willy
Re: Huge performance issues with Haproxy + SSL
On Fri, Oct 11, 2013 at 02:43:35AM +, steve wrote: I have been working on trouble shooting Haproxy 1.5 dev 19 with SSL for the last day or so on Cent OS 6.4 64bit. Lastest OpenSSL compiled 1.0.1e, recompiled haproxy with this make -s TARGET=linux2628 USE_EPOLL=1 USE_OPENSSL=1 ARCH=x86_64 clean all SSL cert wild card, plus godaddy intermediate and our key. Our current set of issues we are seeing: *Massive amounts of connection refused when running the test with ssl *Very High usage of CPU on this 8 core 32 gig box with 100 gig ssd and 1gb nic *Maybe 1/4 the amount of traffic we can push though, compaired to a non ssl test We are using Jmeter to load test and blazemeter to do up to 40k jmeter threads for a full hour. Are you sure your haproxy settings support these 40k concurrent connections ? Here is a list of the errors that are spit back after the test is done Response codes response codecountresponse message 400 29 Bad request This means SSL could pass through but that it's the tester which is sending bad requests. Quite concerning in fact because from this point it's permitted to doubt about everything else... Non HTTP response code: javax.net.ssl.SSLPeerUnverifiedException 86069 Non HTTP response message: peer not authenticated Possibly aborted handshakes. Non HTTP response code: org.apache.http.conn.HttpHostConnectException 27229 Non HTTP response message: Connection to https://.com: refused Huh ? did you stop and restart haproxy during the test ? Are you sure the connectivity between the client and haproxy is OK ? A connection refused can only happen when either the server is stopped or when there is one component between the client and the server which explicitly sends RST packets (eg: a firewall), or some ICMP admin prohibited packets (eg: a router). Non HTTP response code: java.net.SocketException 88 Non HTTP response message: Connection reset 4122 Precondition Failed Non HTTP response code: org.apache.http.NoHttpResponseException270Non HTTP response message: The target server failed to respond So this is what we are facing and we are not haproxy experts and think we have taken it to the best of what we understand about haproxy config and settings. special note: we do not have a web site on the backend, its user server for an upcoming game we are working on so the stack is quite simple from haproxy - node.js -- db and back. Json data is posted to the user server and returned. OK. Anyway this is like a website and it must work! You need to check haproxy's logs to see if it *receives* the requests that are reported to fail, or if it logs failed handshakes. Since you're reporting a high CPU usage, it is also possible that the client renegociates a new key for each request, which might or might not match what you expect from your target. For example, if each of your clients does only one request and leaves, this is OK. But if you have only 40k concurrent clients which do a number of requests, they will only negociate once at the beginning of their session. Regards, Willy
Re: Need help with 1.5 crashing when browser refreshed
Hi Kevin, On Thu, Oct 10, 2013 at 08:28:07PM -0500, Kevin wrote: So after some more troubleshooting the problem seems to be related to PCRE. If I compile without it I don?t see the problem. In 1.4 there does not appear to be any issue using PCRE. OK, thanks for tracking this down. I'm not seeing changes specific to PCRE in 1.5 except the support for the JIT version which is not enabled by default (you need USE_PCRE_JIT for this). Just a quick question, are you using the GNU make file (Makefile) or the OSX Makefile (Makefile.osx) to build haproxy ? Could you please send the complete output of haproxy -vv ? It's also possible that you're hitting a completely different bug that is triggerred by the use of PCRE but not related to it (eg: a use after free or something like this). Thanks, Willy
