I have a big desire to never over complicate things. I've decided that each backend server will be configured identically and will set themselves up by the Public IP assigned to the server. I might just start out with Amazon's ELB and Auto-Scale to load balance and spin up/down server instances as needed. I will use HAProxy in the backend servers and will see if I can script dynamic configuration of HAProxy for load balancers instead of ELB (since I'd like to be able to move to colo easily when the time comes and not introduce new software into my production software stack).
I do plan to use DNSMadeEasy DNS load balancing (which has server monitoring and failover built in) to do an initial balancing of the requests. This might mean that I can completely eliminate the need for frontend HAProxy instances and simplify my backend server set up even more (still having HAProxy in the backends to do cookie persistence and guarding for overloading the localhost nginx/php/mysql backend). But, at this point I still believe I need HAProxy frontend servers to increase availability (so backend servers can be rebooted with no frontend downtime). As a smaller user of HAProxy, I do wish dynamic configuration was better supported in the base HAProxy and that it was documented thoroughly what the "best practices" are for setting up load balancers in front of a dynamically expanding/collapsing farm of identical backend servers. Kevin On Jan 13, 2013, at 5:47 PM, Julien Vehent <[email protected]> wrote: > > That's how we do it at AWeber. The chef-client uses the chef-server to find > all backends, generate a configuration file, and reloads haproxy if the conf > is different than the current one. > Chef and puppet make it so easy to add servers, and auto configure them, that > I can't find any reason why you'd want to edit the haproxy conf manually. > Automate the whole thing instead! > > - Julien > > > > -----Original Message----- > From: Baptiste <[email protected]> > To: KT Walrus <[email protected]> > Cc: Zachary Stern <[email protected]>, [email protected] > Sent: Sat, 12 Jan 2013 7:10 > Subject: Re: Backend Server Dynamic Configuration > > I confirm, I saw many people using chef to mange their haproxy > configuration. > > cheers > > > On Thu, Jan 10, 2013 at 12:30 AM, KT Walrus <[email protected]> wrote: > >> Zachary, >> >> I guess I'll have to spend some time researching puppet (and/or chef). >> Thanks for the reference. >> >> Kevin >> >> On Jan 9, 2013, at 6:18 PM, Zachary Stern <[email protected]> wrote: >> >> Case in point for why puppet (and probably also chef) is perfect here. >> >> You can manage the config with puppet, and have the service "subscribe" to >> the config file, so that it autorestarts or reloads every time puppet >> changes is. >> >> >> On Wed, Jan 9, 2013 at 6:14 PM, Willy Tarreau <[email protected]> wrote: >> >>> Hi Kevin, >>> >>> On Wed, Jan 09, 2013 at 04:13:28PM -0500, Kevin Heatwole wrote: >>> (...) >>>> 1. Setting new interval time for subsequent configuration checks to >>> the server. >>>> 2. Setting new maxconn or weight for the server (allowing backend to >>> "throttle" or "increase" load for itself). >>>> 3. Setting server state (DOWN, MAINTENANCE, UP, STARTING, STOPPING, >>> DAMAGED) changing how HAProxy treats existing connections or new >>> connections for the server. >>>> 4. Change server from active to backup (or vice versa). >>>> 4. Adding new server to backend by having configuration check return >>> new server configuration. >>>> 5. Changing any other useful settings that affect backend servers. >>> >>> This is more or less what was spec'd by Malcolm recently and implemented >>> by Simon. We found that some details could be adapted before merging, but >>> in general it's very close. >>> >>> Anyway you won't hot-add a server that soon : lots of internal sizes >>> (including file descriptor limits) depend on the number of servers. At >>> most we'd have provisions for adding up to N extra servers. Also, a lot >>> of checks are currently performed on server configuration (eg: don't >>> track a server that tracks yourself). Doing some of these checks at >>> runtime and providing enough feedback to the user to react will be quite >>> tricky. >>> >>> In practice, you can already tweak a number of things on the stats CLI. >>> I'd like to be able to switch a server's state betwen active/backup, but >>> it's already algorithmically complex. Changing an address would be really >>> nice, but comes with issues again. >>> >>> Anyway, whatever changes you make, you'll have to do them on the config >>> file too because if you restart your process, you don't want all these >>> changes to be lost ! So right now changing the config only and doing a >>> reload is already a reasonable tradeoff. >>> >>> Willy >>> >>> >>> >> >> >> -- >> >> zachary alex stern I systems architect >> >> o: 212.363.1654 x106 | f: 212.202.6488 | [email protected] >> >> 60-62 e. 11th street, 4th floor | new york, ny | 10003 >> >> www.enternewmedia.com >> >> >> >
