haproxy and weight
Hi, I did some tests with HAProxy. It didn't act as I expected, may be my expectation was wrong. I have 3 servers that are hammering, with siege, a HAProxy ( 1.4.8) with two apache servers behind it. The haproxy is configured with balance roundrobin, cookie JSESSIONID prefix and stats socket configured to control the HAProxy via hatop. When I change the weight of one of the servers to 0%. New connections still end up at this server from the server that was ending up at this server are still forwarded to this server. Only when I disable the server via hatop I see new connections moving to the other server, regardless of cookies, and an active connection like a download can continue until it has completed. When the prefix JSESSIONID has been set and the server connected to the prefix goes down the connection is moved to the other server. when the cookie doesn't get changed the connection is moved back right away when the server is up again. I expected the following of the commands. Change weight to 0% all connections except the active ones (still downloading and with a cookie that show that certain server should be used) move to the other server(s). A disable server cuts all connections also the ones that are still active. The reason I ask this because this way it would be the least service effecting if you need to do some maintenance. Set weight to 0% Check how many active sessions in the application server. If the amount is low enough, disable the application server via hatop. Did I do something wrong, is this related to this server version, is an issue that still exists? Regards, Joop.
Re: Need more info on compression
On 24.11.2012 18:25, Willy Tarreau wrote: Hi Dmitry, On Thu, Nov 22, 2012 at 08:03:26PM +0400, Dmitry Sivachenko wrote: Hello! I was reading docs about HTTP compression support in -dev13 and it is a bit unclear to me how it works. Imagine I have: compression algo gzip compression type text/html text/javascript text/xml text/plain in defaults section. What will haproxy do if: 1) backend server does NOT support compression; Haproxy will compress the matching responses. 2) backend server does support compression; You have two possibilities : - either you just have the lines above, and the server will see the Accept-Encoding header from the client and will compress the response ; in this case, haproxy will see the compressed response and will not compress again ; - or you also have a compression offload line. In this case, haproxy will remove the Accept-Encoding header before passing the request to the server. The server will then *not* compress, and haproxy will compress the response. This is what I'm doing at home because the compressing server is bogus and sometimes emits wrong chunked encoded data! 3) backend server does support compression and there is no these two compression* lines in haproxy config. Then haproxy's normal behaviour remains unchanged, the server compresses if it wants to and haproxy transfers the response unmodified. I think documentation needs to clarify things a bit. Possibly, however I don't know what to clarify nor how, it's always difficult to guess how people will understand a doc :-( Could you please propose some changes ? I would be happy to improve the doc if it helps people understand it. Thank you very much for the explanation. Please consider the attached patch, I hope it will clarify haproxy's behavior a bit. --- configuration.txt.orig 2012-11-26 06:11:05.0 +0400 +++ configuration.txt 2012-11-28 17:45:25.0 +0400 @@ -1903,16 +1903,23 @@ Compression will be activated depending on the Accept-Encoding request header. With identity, it does not take care of that header. + If backend servers support HTTP compression, these directives + will be no-op: haproxy will see the compressed response and will not + compress again. If backend servers do not support HTTP compression and + there is Accept-Encoding header in request, haproxy will compress the + matching response. The offload setting makes haproxy remove the Accept-Encoding header to prevent backend servers from compressing responses. It is strongly recommended not to do this because this means that all the compression work will be done on the single point where haproxy is located. However in some deployment scenarios, haproxy may be installed in front of a buggy gateway - and need to prevent it from emitting invalid payloads. In this case, simply - removing the header in the configuration does not work because it applies - before the header is parsed, so that prevents haproxy from compressing. The - offload setting should then be used for such scenarios. + with broken HTTP compression implementation which can't be turned off. + In that case haproxy can be used to prevent that gateway from emitting + invalid payloads. In this case, simply removing the header in the + configuration does not work because it applies before the header is parsed, + so that prevents haproxy from compressing. The offload setting should + then be used for such scenarios. Compression is disabled when: * the server is not HTTP/1.1.
Re: problem with sort of caching of use_backend with socket.io and apache
Thanks willy, i solved it as soon you answer me but i'm still dealing
to the configuration to make it work as i need:
my last question was this:
http://serverfault.com/questions/451690/haproxy-is-caching-the-forwarding
and i got it working, but for some reason, after the authentication is
made and the some commands are sent, the connection is dropped and a
new connection is made as you can see here:
info - handshake authorized 2ZqGgU2L5RNksXQRWuhi
debug - setting request GET /socket.io/1/websocket/2ZqGgU2L5RNksXQRWuhi
debug - set heartbeat interval for client 2ZqGgU2L5RNksXQRWuhi
debug - client authorized for
debug - websocket writing 1::
debug - websocket received data packet
5:3+::{name:ferret,args:[tobi]}
debug - sending data ack packet
debug - websocket writing 6:::3+[woot]
info - transport end (socket end)
debug - set close timeout for client 2ZqGgU2L5RNksXQRWuhi
debug - cleared close timeout for client 2ZqGgU2L5RNksXQRWuhi
debug - cleared heartbeat interval for client 2ZqGgU2L5RNksXQRWuhi
debug - discarding transport
debug - client authorized
info - handshake authorized WkHV-B80ejP6MHQTWuhj
debug - setting request GET /socket.io/1/websocket/WkHV-B80ejP6MHQTWuhj
debug - set heartbeat interval for client WkHV-B80ejP6MHQTWuhj
debug - client authorized for
debug - websocket writing 1::
debug - websocket received data packet
5:4+::{name:ferret,args:[tobi]}
debug - sending data ack packet
debug - websocket writing 6:::4+[woot]
info - transport end (socket end)
i tried several configurations, something like this:
http://stackoverflow.com/questions/4360221/haproxy-websocket-disconnection/
and also declaring 2 backends, and using ACL to forward to a backend
that has the
option http-pretend-keepalive
when the request is a websocket request and to a backend that has
http-server-close when the request is only for socket.io static files
or is any other type of request that is not websocket.
i would clarify that http-server-close is only on the nginx backend
and in the static files backend, http-pretend-keepalive is on frontend
all and in the websocket backend.
anyone could point me to the right direction? i tried several
combinations and none worked so far :(
thanks in advance for your time and patience :)
2012/11/24 Willy Tarreau w...@1wt.eu:
Hi David,
On Sat, Nov 24, 2012 at 09:26:56AM -0300, david rene comba lareu wrote:
Hi everyone,
i'm little disappointed with a problem i'm having trying to configure
HAproxy in the way i need, so i need a little of help of you guys,
that knows a lot more than me about this, as i reviewed all the
documentation and tried several things but nothing worked :(.
basically, my structure is:
HAproxy as frontend, in 80 port - forwards by default to webserver
(in this case is apache, in other machines could be nginx)
- depending the domain
and the request, forwards to an Node.js app
so i have something like this:
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
maxconn 4096
user haproxy
group haproxy
daemon
defaults
log global
modehttp
maxconn 2000
contimeout 5000
clitimeout 5
srvtimeout 5
frontend all 0.0.0.0:80
timeout client 5000
default_backend www_backend
acl is_soio url_dom(host) -i socket.io #if the request contains socket.io
acl is_chat hdr_dom(host) -i chaturl #if the request comes from chaturl.com
use_backend chat_backend if is_chat is_soio
backend www_backend
balance roundrobin
option forwardfor # This sets X-Forwarded-For
timeout server 5000
timeout connect 4000
server server1 localhost:6060 weight 1 maxconn 1024 check #forwards to
apache2
backend chat_backend
balance roundrobin
option forwardfor # This sets X-Forwarded-For
timeout queue 5
timeout server 5
timeout connect 5
server server1 localhost:5558 weight 1 maxconn 1024 check #forward to
node.js app
my application uses socket.io, so anything that match the domain and
has socket.io in the request, should forward to the chat_backend.
The problem is that if i load directly from the browser, let say, the
socket.io file (it will be something like
http://www.chaturl.com/socket.io/socket.io.js) loads perfectly, but
then when i try to load index.html (as
http://www.chaturl.com/index.html) most of the times, is still
redirect to socket.io. after refreshing a few time, it finally loads
index.html, but then, doesn't load the socket.io.js file inserted in
the file (why it redirect to the apache server, and not the node.js
app). so as i said, it sort of caching the request.
i tried several ACL combinations, i disabled the domain check, only
checking for socket.io but is still the same. Reading again the
documentation i tried to use hdr_dir, hdr_dom,
stunnel + haproxy + ssl + ddns + multiple domains
All, wondering if you can point me in the right direction. I have stunnel installed with the x-forwarded-for patch. I also have haproxy working so all incoming http requests are forwarded from my router to happroxy. haproxy then determines where to route the request based on the domain name. Configs below. I'd like to implement something similar with stunnel and haproxy so that all inbound requests can be routed in the same manner for https. global log 127.0.0.1 local2 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 4000 userhaproxy group haproxy daemon # turn on stats unix socket stats socket /var/lib/haproxy/stats defaults modehttp log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 3000 frontend http_proxy bind *:80 acl is_rbc-com hdr_dom(host) -i robcluett.com acl is_rbc-net hdr_dom(host) -i robcluett.net acl is_iom-com hdr_dom(host) -i iomerge.com use_backend cluster1 if is_rbc-com use_backend cluster2 if is_rbc-net use_backend cluster3 if is_iom-com backend cluster1 server web2 10.10.10.51:80 #server web5 192.168.1.128 backend cluster2 server web3 10.10.10.52:80 #server web6 192.168.1.129:80 backend cluster3 server web4 10.10.10.53:80 Rob Cluett r...@robcluett.com 978.381.3005 **Please use this address for all email correspondence. The phone number listed in the signature above replaces any other phone number you may have for me.* *This email contains a digitally signed certificate authenticating the sender. This certificate prevents others from posing as or spoofing the sender, guarantees that it was sent from the named sender and when necessary encrypts the email such that only the sender and reciepient(s) can read it's contents. If you receive an email from this sender without the digitally signed certificate it is not from the sender and therefore it's contents should be disregarded. * * * *This e-mail, and any files transmitted with it, is intended solely for the use of the recipient(s) to whom it is addressed and may contain confidential information. If you are not the intended recipient, please notify the sender immediately and delete the record from your computer or other device as its contents may be confidential and its disclosure, copying or distribution unlawful.*** smime.p7s Description: S/MIME cryptographic signature
Re: stunnel + haproxy + ssl + ddns + multiple domains
Hi Rob, Just make you stunnel point to your frontend on the port 80, and you're done. cheers On Thu, Nov 29, 2012 at 1:05 AM, Rob Cluett r...@robcluett.com wrote: All, wondering if you can point me in the right direction. I have stunnel installed with the x-forwarded-for patch. I also have haproxy working so all incoming http requests are forwarded from my router to happroxy. haproxy then determines where to route the request based on the domain name. Configs below. I'd like to implement something similar with stunnel and haproxy so that all inbound requests can be routed in the same manner for https. global log 127.0.0.1 local2 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 4000 userhaproxy group haproxy daemon # turn on stats unix socket stats socket /var/lib/haproxy/stats defaults modehttp log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 3000 frontend http_proxy bind *:80 acl is_rbc-com hdr_dom(host) -i robcluett.com acl is_rbc-net hdr_dom(host) -i robcluett.net acl is_iom-com hdr_dom(host) -i iomerge.com use_backend cluster1 if is_rbc-com use_backend cluster2 if is_rbc-net use_backend cluster3 if is_iom-com backend cluster1 server web2 10.10.10.51:80 #server web5 192.168.1.128 backend cluster2 server web3 10.10.10.52:80 #server web6 192.168.1.129:80 backend cluster3 server web4 10.10.10.53:80 Rob Cluett r...@robcluett.com 978.381.3005 *Please use this address for all email correspondence. The phone number listed in the signature above replaces any other phone number you may have for me. This email contains a digitally signed certificate authenticating the sender. This certificate prevents others from posing as or spoofing the sender, guarantees that it was sent from the named sender and when necessary encrypts the email such that only the sender and reciepient(s) can read it's contents. If you receive an email from this sender without the digitally signed certificate it is not from the sender and therefore it's contents should be disregarded. This e-mail, and any files transmitted with it, is intended solely for the use of the recipient(s) to whom it is addressed and may contain confidential information. If you are not the intended recipient, please notify the sender immediately and delete the record from your computer or other device as its contents may be confidential and its disclosure, copying or distribution unlawful.
Re: problem with sort of caching of use_backend with socket.io and apache
Hi David,
For more information about HAProxy and websockets, please have a look at:
http://blog.exceliance.fr/2012/11/07/websockets-load-balancing-with-haproxy/
It may give you some hints and point you to the right direction.
cheers
On Wed, Nov 28, 2012 at 6:34 PM, david rene comba lareu
shadow.of.sou...@gmail.com wrote:
Thanks willy, i solved it as soon you answer me but i'm still dealing
to the configuration to make it work as i need:
my last question was this:
http://serverfault.com/questions/451690/haproxy-is-caching-the-forwarding
and i got it working, but for some reason, after the authentication is
made and the some commands are sent, the connection is dropped and a
new connection is made as you can see here:
info - handshake authorized 2ZqGgU2L5RNksXQRWuhi
debug - setting request GET /socket.io/1/websocket/2ZqGgU2L5RNksXQRWuhi
debug - set heartbeat interval for client 2ZqGgU2L5RNksXQRWuhi
debug - client authorized for
debug - websocket writing 1::
debug - websocket received data packet
5:3+::{name:ferret,args:[tobi]}
debug - sending data ack packet
debug - websocket writing 6:::3+[woot]
info - transport end (socket end)
debug - set close timeout for client 2ZqGgU2L5RNksXQRWuhi
debug - cleared close timeout for client 2ZqGgU2L5RNksXQRWuhi
debug - cleared heartbeat interval for client 2ZqGgU2L5RNksXQRWuhi
debug - discarding transport
debug - client authorized
info - handshake authorized WkHV-B80ejP6MHQTWuhj
debug - setting request GET /socket.io/1/websocket/WkHV-B80ejP6MHQTWuhj
debug - set heartbeat interval for client WkHV-B80ejP6MHQTWuhj
debug - client authorized for
debug - websocket writing 1::
debug - websocket received data packet
5:4+::{name:ferret,args:[tobi]}
debug - sending data ack packet
debug - websocket writing 6:::4+[woot]
info - transport end (socket end)
i tried several configurations, something like this:
http://stackoverflow.com/questions/4360221/haproxy-websocket-disconnection/
and also declaring 2 backends, and using ACL to forward to a backend
that has the
option http-pretend-keepalive
when the request is a websocket request and to a backend that has
http-server-close when the request is only for socket.io static files
or is any other type of request that is not websocket.
i would clarify that http-server-close is only on the nginx backend
and in the static files backend, http-pretend-keepalive is on frontend
all and in the websocket backend.
anyone could point me to the right direction? i tried several
combinations and none worked so far :(
thanks in advance for your time and patience :)
2012/11/24 Willy Tarreau w...@1wt.eu:
Hi David,
On Sat, Nov 24, 2012 at 09:26:56AM -0300, david rene comba lareu wrote:
Hi everyone,
i'm little disappointed with a problem i'm having trying to configure
HAproxy in the way i need, so i need a little of help of you guys,
that knows a lot more than me about this, as i reviewed all the
documentation and tried several things but nothing worked :(.
basically, my structure is:
HAproxy as frontend, in 80 port - forwards by default to webserver
(in this case is apache, in other machines could be nginx)
- depending the domain
and the request, forwards to an Node.js app
so i have something like this:
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
maxconn 4096
user haproxy
group haproxy
daemon
defaults
log global
modehttp
maxconn 2000
contimeout 5000
clitimeout 5
srvtimeout 5
frontend all 0.0.0.0:80
timeout client 5000
default_backend www_backend
acl is_soio url_dom(host) -i socket.io #if the request contains socket.io
acl is_chat hdr_dom(host) -i chaturl #if the request comes from chaturl.com
use_backend chat_backend if is_chat is_soio
backend www_backend
balance roundrobin
option forwardfor # This sets X-Forwarded-For
timeout server 5000
timeout connect 4000
server server1 localhost:6060 weight 1 maxconn 1024 check #forwards to
apache2
backend chat_backend
balance roundrobin
option forwardfor # This sets X-Forwarded-For
timeout queue 5
timeout server 5
timeout connect 5
server server1 localhost:5558 weight 1 maxconn 1024 check #forward to
node.js app
my application uses socket.io, so anything that match the domain and
has socket.io in the request, should forward to the chat_backend.
The problem is that if i load directly from the browser, let say, the
socket.io file (it will be something like
http://www.chaturl.com/socket.io/socket.io.js) loads perfectly, but
then when i try to load index.html (as
http://www.chaturl.com/index.html) most of the times, is still
redirect to socket.io. after refreshing a few time, it finally loads
HAProxy in front of Oracle database
Dear List, I've been happily using HAProxy for several years on a variety of scenarios however I've been unable to find any references online with regard to placing HAProxy in front of Oracle database. From my understanding this should be possible (and I've done this with MySQL) but I would like to ask if any of you has any comments/insights/recommendations and/or could point me to a document that covers this. THANK YOU SO MUCH -- unai
