Re: Connection limiting Sorry servers
Hi Willy. First, thank you for your answer. I really don't know why you're limiting on the number of requests per second. It is not the proper way to do this at all. In fact, you should *only* need to play with the server's maxconn parameter, as there should be no reason your servers would be sensible to a number of requests per second. I don't know why limiting number of requests per second wouldn't be proper way to limit the traffic? During high load, we can get few thousand requests per second. Servers can't handle that. If I only set max connections for every server, they won't even get there. With for example 50 new connections at the same time per server, I think they will die before reaching the limit. The thing is we usually have events that start at a certain time and users know that. The event is also disabled until that time so sometimes we really get huge amount of connections at the beginning. If there is some mechanism inside HAProxy, that would help us survive such cases, please let me know. Besides, the contractor thinks that getting waiting room is better user experience than waiting for site to open. That is why I wished to let certain amount of people to site and show everybody else waiting room. Well you could use cookies to know if a user has a sesion on the site or not, but it's still not the right solution. Can someout with more experience than me advice what would be the best way to handle this? I would like to limit number of current users on real servers and amount of new connections that can be created per some time unit. This is the point I cannot agree with. I think you need to limit the amount of concurrent connections on your servers. Otherwise, your application is deadly broken, but that's not how you introduced it first :-) If deadly broken means it does not take care of connection limiting, then it is deadly broken. Application does not do any limiting, servers do when they stop responding :) I would like HAProxy to help me with that as much as possible. This is precisely where the splitting of the backend helps. Assuming that your servers can handle, say, 200 concurrent connections, you split your backends either per application or per application group (you might want to guarantee a quality of service for a group of applications). Then you adjust each server's maxconn so that the total does not exceed the real server's maxconn (=MaxClients on apache). That way, even if an application takes a lot of resources, it will not saturate the server. Also, just enable health-checks in one backend and configure the other ones to track the first one, it will greatly reduce the number of health-checks sent to your servers. If our servers handle can 200 concurrent connections and I limit all sites so that the total doesn't exceed that, I have to limit each site to about 5 concurrent connections (about 40 sites at the moment). That means that instead of using as much resources as possible, the site with the most traffic would be using 5 connections only. Am I right? I am thinking about fullconn option now and since we make sure that there is only one site having high traffic at the moment, I might be able to use that? Thank you and best regards Bostjan
Re: Connection limiting Sorry servers
Hi Bostjan, On Mon, Aug 03, 2009 at 08:51:09AM +0200, Bo??tjan Mer??un wrote: I really don't know why you're limiting on the number of requests per second. It is not the proper way to do this at all. In fact, you should *only* need to play with the server's maxconn parameter, as there should be no reason your servers would be sensible to a number of requests per second. I don't know why limiting number of requests per second wouldn't be proper way to limit the traffic? yes it precisely is a way of limiting the traffic. During high load, we can get few thousand requests per second. Servers can't handle that. why are you saying that ? Except for rare cases of huge bugs, a server is not limited in requests per second. At full speed, it will simply use 100% of the CPU, which is why you bought it after all. When a server dies, it's almost always because a limited resource has been exhausted, and most often this resource is memory. In some cases, it may be other limits such as sockets, file descriptors, etc... which cause some unexpected exceptions not to be properly caught. If I only set max connections for every server, they won't even get there. With for example 50 new connections at the same time per server, I think they will die before reaching the limit. Then set the limit lower. If you're sure they can't handle 50 concurrent connections, it's exactly the reason you must set a maxconn below this value to protect them. The thing is we usually have events that start at a certain time and users know that. The event is also disabled until that time so sometimes we really get huge amount of connections at the beginning. I'm well aware of the problem, many sites have the same. The queuing mechanism in haproxy was developped exactly for that. The first user was a gaming site which went from 50 req/s to 1 req/s on patch days. They too thought their servers could not handle that, while it was just a matter of concurrent connections once again. By enabling the queueing mechanism, they could sustain the 1 req/s with only a few hundred concurrent connections. If there is some mechanism inside HAProxy, that would help us survive such cases, please let me know. Besides, the contractor thinks that getting waiting room is better user experience than waiting for site to open. with the queue, you're not particularly waiting for the site to open, you access it normally, it's just that you don't rush on it all at once. That is why I wished to let certain amount of people to site and show everybody else waiting room. The problem with the waiting room is that only the application knows how many people are on the site. You can't guess that from TCP connection counts. And unfortunately, you will randomly accept then reject users inside a same session. And by doing that, sometimes you'll send some users in the waiting room while there are free slots on the servers. However, with the queue you have the ability to decide to send people to a waiting room when the queue is full and if the user has no application cookie for instance. That way you know it's a new user and you prefer to let him wait for the best moment. I would like to limit number of current users on real servers and amount of new connections that can be created per some time unit. This is the point I cannot agree with. I think you need to limit the amount of concurrent connections on your servers. Otherwise, your application is deadly broken, but that's not how you introduced it first :-) If deadly broken means it does not take care of connection limiting, then it is deadly broken. No, being dependant on connection limiting is normal. Being dependant on connections per second is abnormal. Application does not do any limiting, servers do when they stop responding :) Once again I'm well aware of this problem :-) If our servers handle can 200 concurrent connections and I limit all sites so that the total doesn't exceed that, I have to limit each site to about 5 concurrent connections (about 40 sites at the moment). That means that instead of using as much resources as possible, the site with the most traffic would be using 5 connections only. No, if your servers are shared between multiple sites, you can very well use a same backend for all those sites, or even for some groups of sites. Also, limiting on the number of requests per second will never prevent a slow site from saturating your 200 concurrent connections. Am I right? I am thinking about fullconn option now and since we make sure that there is only one site having high traffic at the moment, I might be able to use that? The fullconn is only used in conjunction with the minconn, it only tells the algorithm what threshold must be considered full load. Hoping this helps, Willy
Question about TCP balancing
Hello! I am trying to setup haproxy 1.3.19 to use it as TCP load balancer. Relevant portion of config looks like: listen test 0.0.0.0:17000 mode tcp balance roundrobin server srv1 srv1:17100 check inter 2 server srv2 srv2:17100 check inter 2 server srv3 srv3:17100 check inter 2 Now imagine the situation that all 3 backends are down (no program listen on 17100 port, OS responds with Connection Refused). In that situation haproxy still listens port 17100 and closes connection immediately: telnet localhost 17101 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. Connection closed by foreign host. Is it possible to configure haproxy so it will stop listening the port when all backends are down? So clients will receive Connection Refused as if none listens TCP port at all? Thanks in advance!
Re: Question about TCP balancing
Hello Dmitry, On Mon, Aug 03, 2009 at 02:24:47PM +0400, Dmitry Sivachenko wrote: Hello! I am trying to setup haproxy 1.3.19 to use it as TCP load balancer. Relevant portion of config looks like: listen test 0.0.0.0:17000 mode tcp balance roundrobin server srv1 srv1:17100 check inter 2 server srv2 srv2:17100 check inter 2 server srv3 srv3:17100 check inter 2 Now imagine the situation that all 3 backends are down (no program listen on 17100 port, OS responds with Connection Refused). In that situation haproxy still listens port 17100 and closes connection immediately: telnet localhost 17101 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. Connection closed by foreign host. Is it possible to configure haproxy so it will stop listening the port when all backends are down? So clients will receive Connection Refused as if none listens TCP port at all? No it's not, and it's not only a configuration issue, it's an OS limitation. The only way to achieve this is to stop listening to the port then listen again to re-enable the port. On some OSes, it is possible. On other ones, you have to rebind (and sometimes close then recreate a new socket). But once your process has dropped privileges, you can't always rebind if the port is 1024 for instance. So instead of having various behaviours for various OSes, it's better to make them behave similarly. I have already thought about adding an OS-specific option to do that, but I have another problem with that. Imagine that your servers are down. You stop listening to the port. At the same time, someone else starts listening (eg: you start a new haproxy without checking the first one, or an FTP transfer uses this port, ...). What should be done when the servers are up again ? Haproxy will not be able to get its port back because someone else owns it. So, by lack of a clean and robust solution, I prefer not to experiment in this area. Regards, Willy
