> This is the role of the first layer of load-balancer. > The maxconn features make it smart: as soon as one of your backend > server reaches its maxconn, it's pulled out from the LB algorithm > untill the number of connections decrease. > I definitively would not use 2 layers of HAProxy and would not use > first algo too…
I don't quite understand. Are you saying that if I have 2 or more LBs routing requests to the same backends, that each LB knows how many requests are actually being serviced by each backend (including counting the requests to a backend from different LBs). I really want my architecture to have multiple active LBs so the architecture will infinitely scale. > note that, an active/active HAProxy can be designed (with tricked > configuration) to adapt its maxconn based on the number of LBs > available. Is this documented anywhere? I would like to understand how to set this up. In the meantime, I don't have a problem with having a "guard" HAProxy in each backend. I'll have to have an HAProxy instance to handle the HA for the sessionDBs anyway, so it shouldn't be that much more to have the extra "hop" in going through the "guard". > no, you said your domain name would be propagated over 2 dns A entries... > so a client could arrive at any time on any LB (modulo the DNS TTL) Yes. I understand that. This is why the architecture doesn't require "session stickiness" to a particular backend. It just requires that the "session" use the same backend sessionDB for all requests. So, I hope to have 1 A record in the DNS for each active frontend LB and I hope to be able to scale to as many frontend LBs as needed for all possible loads. > To avoid loosing traffic, HAProxy could also be used to forward > traffic to the other DC if all the server in its local DC are > unavailable. Actually, if all backend servers in its local DC are unavailable, the frontend HAProxy should "down itself". This way, the DNS Failover will kick in and all subsequent requests (after the DNS TTL refreshes) will go to the other DC. The frontend HAProxy will only need to forward traffic to the other DC until the DNS Failover has kicked in for all users. It is more likely that some core DC routers fail or a power outage at the DC knocks all servers offline, both frontend LBs and backends. In this case, you really need the DNS Failover to route all traffic to the surviving DC to minimize user perceived downtime (at DNSMadeEasy, they say that Failover can happen fully in under 5 minutes if TTLs are set low and DNS caching respects TTLs). Kevin On Jan 3, 2013, at 3:45 AM, Baptiste <[email protected]> wrote: > On Thu, Jan 3, 2013 at 8:56 AM, KT Walrus <[email protected]> wrote: >> Thanks for the reply. >> >>> Why installing 2 layers of HAProxy??? >>> A single one (on the 2 servers is enough). >> >> My thought was that the second layer of HAProxy would ensure that the >> individual backend server would never have more than MAXCONN requests so I >> know the server will never be overloaded possibly leading to the server >> going down or taking too long to process a request. >> > cf below > >>> I'm not a fan of first algo, unless you pay the resource per number of >>> backend server, which is not your case. >> >> I just thought "first" load balancing was perfect for "guarding" that an >> individual backend server never exceeded MAXCONN concurrent requests. The >> overhead should be minimal since this "guard" HAProxy almost always will >> pass the request to localhost nginx/varnish. I need this "guard" because >> there are multiple frontend LBs doing simple round robin to the backends >> independently. This might become more of a possibility when and if I need >> more LBs independently distributing requests to the backends. >> > > This is the role of the first layer of load-balancer. > The maxconn features make it smart: as soon as one of your backend > server reaches its maxconn, it's pulled out from the LB algorithm > untill the number of connections decrease. > I definitively would not use 2 layers of HAProxy and would not use > first algo too... > > note that, an active/active HAProxy can be designed (with tricked > configuration) to adapt its maxconn based on the number of LBs > available. > > >>> Prefer using a hash in your case (even multiple hash with different >>> backends and content switching), that way, your hit rate would be much >>> better. >> >> I'm not so concerned about individual hit rate as I am about HA and infinite >> scalability. It is relatively cheap to add a new server to handle more >> backend or frontend load or split to placing some servers in a new >> datacenter. I'd rather have my servers run at 50% capacity (purchasing >> twice the hardware) if that means increased HA from having the guard >> HAProxy's and never coming close to pushing them too hard that individual >> pieces of the software/hardware stack start to fail. >> > > This is not a question of cost, this is a question of efficiency and > response time (on the client side) > >>> An other remark, it may be hard to troubleshoot such infra with 2 >>> Active/active LBs. >> >> I think I have to deal with this, but since each LB is handling unique VIPs >> (unless keepalived kicks in due to failure), I don't think there is going to >> be that much trouble. > > no, you said your domain name would be propagated over 2 dns A entries... > so a client could arrive at any time on any LB (modulo the DNS TTL) > >>> And using DNS rr does not prevent you from using keepalived to ensure >>> HA between your 2 HAProxys. >> >> Yes. I am hoping this is the case. I eventually want at least two >> geographic locations (east and west coast data centers) so 4 IPs in the DNS >> to distribute to the closest datacenter. I use DNSMadeEasy which can >> support both DNS Global Traffic Director (east coast and west coast IP >> Anycast) and DNS Failover (incase one datacenter goes offline). >> > > That is good. > To avoid loosing traffic, HAProxy could also be used to forward > traffic to the other DC if all the server in its local DC are > unavailable. > >>> >>> cheers >>> >>> >>> On Thu, Jan 3, 2013 at 12:20 AM, KT Walrus <[email protected]> wrote: >>>> I'm setting up a new website in the next month or two. Even though the >>>> traffic won't require a scalable HA website, I'm going to start out as if >>>> the website needs to support huge traffic so I can get some experience >>>> running such a website. >>>> >>>> I'd like any feedback on what I am thinking of doing… >>>> >>>> As for hardware, I am colocating 6 servers at this time and plan to use >>>> Amazon S3 to host the static files (which should grow quickly to 1TB or >>>> 2TB of mostly images). 2 of the servers are going to be my frontend load >>>> balancers running haproxy. The remaining 4 servers with be nginx/varnish >>>> servers (nginx for the PHP/MySQL part of the site and varnish to cache the >>>> Amazon S3 files to save bandwidth charges by Amazon). >>>> >>>> I plan on doing DNS load balancing using pairs of A records for each >>>> hosted domain that will point to each of my frontend haproxy load >>>> balancers. Most traffic will be HTTPS, so I plan on having the frontend >>>> load balancers to handle the SSL (using the new haproxy support for SSL). >>>> >>>> The two load balancers will proxy to the 4 backend servers. These 4 >>>> backend servers will run haproxy in front of nginx/varnish with load >>>> balancing of "first" and a suitable MAXCONN. Server 1 haproxy will first >>>> route to the localhost nginx/varnish and when MAXCONN connections are >>>> active to the localhost, will forward the connection to Server 2 haproxy. >>>> Server 2 and 3 will be set up similarly to first route requests to >>>> localhost and when full, route subsequent requests to the next server. >>>> Server 4 will route excess requests to a small Amazon EC2 instance to >>>> return a "servers are all busy" page. Hopefully, I will be able to add a >>>> 5th backend server at Amazon to handle the overload if it looks like I >>>> really do have traffic that will fill all 4 backend servers that I am >>>> colo'ing (I don't really expect this to ever be necessary). >>>> >>>> Nginx will proxy to PHP on localhost and each localhost (of my 4 backend >>>> servers) will have 2 MySQL instances - one for the main Read-Only DB and >>>> one for a Read-Write SessionDB. PHP will go directly to the main DB (not >>>> through HAProxy) and will use HAProxy to select the proper SessionDB to >>>> use (each user session must use the same SessionDB so the one a request >>>> needs might be on any of the backend servers). Each SessionDB will be the >>>> master of one slave SessionDB on a different backend server for handling >>>> the failure of the master (haproxy will send requests to the slave >>>> SessionDB if the master is down or failing). >>>> >>>> So, each backend server will have haproxy to "first" balance HTTP to >>>> nginx/varnish. The backends also have PHP and 3 instances of MySQL (one >>>> for mainDB, one for master sessionDB, and one for another backend's slave >>>> sessionDB). >>>> >>>> Also, the 2 frontend servers will be running separate instances of >>>> haproxy. I hope to use keepalived to route the VIPs for one frontend to >>>> the other frontend in case of failure. Or, should I use heartbeat? There >>>> seems to be two HA solutions here. >>>> >>>> I know this is a very long description of what I am thinking of doing and >>>> I thank you if you have read this far. I'm looking for any comments on >>>> this setup. Especially, any comments on using "first" load >>>> balancing/MAXCONN on the backend servers so that a request load balanced >>>> from the frontend will keep the backend servers from overloading (possibly >>>> bouncing a request from server 1 to server 2 to server 3 to server 4 to >>>> EC2 "server busy" server) are especially appreciated. Also, any comments >>>> on using pairs of master/slave sessionDBs to provide high availability but >>>> still have session data saved/retrieved for a given user from the same DB >>>> are appreciated. I believe this setup will allow the load to be >>>> distributed evenly over the 4 backends and only have the front end load >>>> balancers do simple round robin without session stickiness. >>>> >>>> Kevin >>>> >>>> >>
