Re: Load balancers
On Mon, 13 Jan 2003, John Siracusa wrote: Thanks for all the info, and please feel free to send me more, especially if there's some gem of a software load balancer out there somewhere... :) mod_accel can do primitive load balancing (via DNS) and primitive fault tolerance. It connect to next backend 1) if connect() to current backend failed; 2) if it can not get HTTP header from backend (timeout or error); 3) if it received 5XX response from backend (it can be disabled). Nevertheless it does not remember any faults to future use so I called it primitive. You can find it at http://sysoev.ru/en/ If you use DNS-based load balancing then you should install local caching DNS server on the same computer to speed up DNS queries. Igor Sysoev http://sysoev.ru/en/
Load balancers
(This may seem off topic for this list, but I'll try to bring it around before the end of the message :) We've been struggling with load balancers for a while now. My requirements are pretty simple. I have a handful of plain and mod_perl apache servers, some of which are identical and a few of which are the only ones that can perform unique functions. We want a load balancer that: * distributes requests to the server(s) that can handle them. I need to route to specific servers based on requested URL and perhaps headers and/or protocol type (SSL). * detects when a server is down and does not send request to that server. This process should be as transparent to the user as possible. * doesn't crash or lock up. We started with mod_backhand, but it frequently seemed to get confused, failing to send requests to some hosts that were perfectly able to service requests, and then locking up entirely when a back-end server became overloaded. I also found the diagnostics to be inaccurate and misleading (or at least poorly documented), and the documentation in general to be lacking. Next, we tried pound, which looked promising due to its simplicity. Unfortunately, it too seems to freak out at least once a day, filling its log with errors like this: @40003e1ef2461015ce34 warning: error read from 123.123.123.123: Success @40003e1ef24621631c8c warning: cannot read headers. got [] @40003e1ef2462163d424 warning: error read from 123.123.123.123: Success @40003e1ef2462902488c warning: cannot read headers. got [] @40003e1ef246290332ec warning: error read from 123.123.123.123: Success @40003e1ef246312225dc warning: cannot read headers. got [] @40003e1ef2463122be34 warning: error read from 123.123.123.123: Success @40003e1ef24706782a84 warning: cannot read headers. got [] @40003e1ef2470678bef4 warning: error read from 123.123.123.123: Success @40003e1ef247176094bc warning: cannot read headers. got [] We've been in contact with the author and have tried digging around in and/or patching the source code with no improvement so far. But meanwhile, we're still open to alternatives. Surprisingly, there don't seem to be many (software) options. (A hardware load balancer is not an option at his time, but I'll also take any suggestions in that area :) This brings me to the mod_perl angle. Has anyone ever tried using a slimmed down mod_perl server as a load balancer? Is this feasible? Making routing decisions is obviously the easy part when using mod_perl, but would those mod_perl apache process just be too big and too slow to proxy requests efficiently? And how would they deal with detecting back-end servers that have failed? (We're running various versions of Linux on our servers, if that makes any difference.) So...suggestions? How are other people handling load balancing? -John
Re: Load balancers
On 1/13/03 11:12 AM, Ask Bjoern Hansen wrote: On Mon, 13 Jan 2003, John Siracusa wrote: So...suggestions? How are other people handling load balancing? With hardware load balancers. :-) Sure, rub it in ;) You forgot to include the information about number of servers, requests per second at peak times, reponse sizes, etc etc. * four servers * peak of 30 hits/sec (fluctuates though) * response sizes vary: most are less than 20K, but a significant minority are 500K-5MB (i.e. file downloads) But really, stability is out #1 concern right now. Slowdown under load is much more desirable that error under load or lock-up under load :) -John
Re: Load balancers
John Siracusa wrote: But meanwhile, we're still open to alternatives. Surprisingly, there don't seem to be many (software) options. (A hardware load balancer is not an option at his time, but I'll also take any suggestions in that area :) I've always used hardware ones. I believe big/ip does everything you need. However, if I were going to use a software solution I would be looking at Linux Virtual Server, probably starting with the Red Hat offering based on it. This brings me to the mod_perl angle. Has anyone ever tried using a slimmed down mod_perl server as a load balancer? Is this feasible? Not a good idea. It would eat all your memory. You'd be much better off with the mod_rewrite solution mentioned in this thread. - Perrin
Re: Load balancers
On Mon, 13 Jan 2003, John Siracusa wrote:
But meanwhile, we're still open to alternatives. Surprisingly, there don't
seem to be many (software) options. (A hardware load balancer is not an
option at his time, but I'll also take any suggestions in that area :)
My experience in this area led quickly to the conclusion that there is no
viable software solution, at least when you get a bit of traffic.
We quickly moved to hardware load balancing and did very well with the
Coyote Point Equalizer, which was affordable and powerful. Also the
developers (Bill Kish is/was the lead developer) were very responsive and
worked with us to tweak as necessary.
After a while we got one of those loser CFOs who was golf pals with the
new COO or something, and he bought a bunch of Foundry Systems Server
Irons that worked alright but were orders of magnitude more expensive.
- nick
Nick Tonkin {|8^)
Re: Load balancers
On Monday 13 January 2003 08:03 am, John Siracusa wrote: (We're running various versions of Linux on our servers, if that makes any difference.) So...suggestions? How are other people handling load balancing? -John Hey John, Take a look at the LVS project, www.foundrynetworks.com and www.f5networks.com. I've used Foundry and F5 gear for load-balancing with session persistence and had really good results - cost might be an issue with both of these units. I have been putting together an LVS (DR) solution for some work that's similar to what your doing but haven't finished it yet. Steve
Re: Load balancers
John Siracusa wrote: [snip] But meanwhile, we're still open to alternatives. Surprisingly, there don't seem to be many (software) options. (A hardware load balancer is not an option at his time, but I'll also take any suggestions in that area :) [snip] So...suggestions? How are other people handling load balancing? Squid in accelerator mode for ordinary http and the linux virtual server stuff for https. Works nicely for us (about a couple of million requests a day, mostly http). Andy -- Andy Osborne Vertical B2B Communities Senior Internet Engineer Sift Group100 Victoria Street, Bristol BS1 6HZ tel:+44 117 915 9600 fax:+44 117 915 9630 http://www.sift.co.uk
Re: Load balancers
The Load Balancing section of this doc might help: http://httpd.apache.org/docs/misc/rewriteguide.html Just straight apache + mod_rewrite could be the simple solution you seek. The Proxy Throughput Round-Robin example shows how to add a script to do mapping as well - could work for your availability detection requirement. We've also used mod_rewrite in a lightweight reverse proxy as a stopgap when load balancing hardware was not available. One option for detecting live backend servers is to use the prg rewrite map like Paolo says, where a script does some sort of availability check. Another option which we have used is the rnd map where a server is selected randomly from a flat file, where that file can be kept updated by an external (non-Apache) process like a cron job that checks backend server availability (mod_rewrite notices on-the-fly when that file has changed). Larry Leszczynski [EMAIL PROTECTED]
RE: Load balancers
Hi John -- But meanwhile, we're still open to alternatives. Surprisingly, there don't seem to be many (software) options. (A hardware load balancer is not an option at his time, but I'll also take any suggestions in that area :) Why is hardware not an option? Cost? If so, I'd take a look at the system Nick recommended -- the Coyote Point Equalizer: http://www.coyotepoint.com/equalizer.htm A few years ago I bought their E-250. It cost about $3,200, which was nothing compared to the other solutions out there at the time. This load balancing system (LBS) was for a corporate intranet with a user base of 50k to 60k users, with the expectation that it would be capable of handling a third of the total user base at any time. The CPE-250 handled the job with ease. We initially rolled out with four separate Web Application Server machines (LAMP architecture), and a procedure to allow us to add additional machines in four hours, if need be. The system has been in production for about two years. In that time we have had individual servers crash, but the intranet has had less than 15 minutes of downtime since we implemented the CPE. Regarding software solutions, mod_perl or otherwise: I initially looked at this, and ultimately rejected it as an option. No robust systems had emerged at the time. Based on a cost/performance comparison against the CP product I calculated it would cost far more than $3,200 to roll our own system which would provide the configurability, ease of use and reliability of the available hardware solution. FWIW, the Equalizer is implemented as a rack-mountable BSD machine running custom software. The software implements load-balancing via dynamic port-forwarding (*NOT* an HTTP proxy), so it is capable of forwarding any TCP service (SMTP, database, etc.). The GUI is web-based, very easy to use, and includes graphical reporting of activity. Load balancing can be configured to be purely round-robin, but the CPE features a very effective dynamic adaptive load balancing system based on server response time which is very effective. It has been a couple years since I talked to Coyote Point about their product. I would check to see if it is still cost-competitive. HTH, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
Re: Load balancers
Hi John, I use wackamole (http://www.backhand.org/wackamole/) on my (2) front end servers with 6 IP addresses doing Round Robin (RR) DNS. This is suffichent for loadbalencing the light (HTML / IMAGE only) front end. I had a issue with this once, it got the IP address allocation confused, but otherwise it's been running fine for a year. The front end machines proxy pass to a backend App (mod_perl) server, they do this by name e.g. backend-server, we then have a script that monitors the backend-server and alters the /etc/resolv.conf file if server 1 goes down and alters the ip address to our second machine. Our second machine is actually the DB machine so this is just an emergency fall back until the apps server can be brought up again. Thankfully I've never had to test this in anger! Leo more info: front ends run debian on standard compaq rack mount servers backend 2 Sun Solaris boxes, 1 with mod_perl other with MySQL traffic: 4 / mill pages a month, 50 / 60 hits a second peak Lots and lots of cacheing on the app and indexes on the DB.
Re: Load balancers
Perrin Harkins [EMAIL PROTECTED] wrote: John Siracusa wrote: But meanwhile, we're still open to alternatives. Surprisingly, there don't seem to be many (software) options. (A hardware load balancer is not an option at his time, but I'll also take any suggestions in that area :) I've always used hardware ones. I believe big/ip does everything you need. However, if I were going to use a software solution I would be looking at Linux Virtual Server, probably starting with the Red Hat offering based on it. We're currently using a couple of big/ip switches, but don't have web servers behind them yet (using them for smtp and such at the moment). We're looking at using them or one of the switches from NetScaler (netscaler.com) which looked quite impressive. NetScalar is really built for web servers (or so it seems from our meetings with them) while big/ip is a more generic solution. Both big/ip and netscalar allow sessions to be bound to a backend server, iirc, which can be a nice optimization (which we haven't had to take advantage of yet). -- James Smith [EMAIL PROTECTED], 979-862-3725 Texas AM CIS Operating Systems Group, Unix
Re: Load balancers
John Siracusa wrote: We've been struggling with load balancers for a while now. My requirements are pretty simple. I have a handful of plain and mod_perl apache servers, So...suggestions? How are other people handling load balancing? I have tested pen. It's easy to set up and works fine so far. The server has only 70k hits/day. I don't know how will it work under heavy load. The purpose was to keep working if one server was down. Pen is a load balancer for simple TCP-based protocols such as HTTP or SMTP. It allows several servers to appear as one to the outside. It automatically detects servers that are down and distributes clients among the available servers. This gives high availability and scalable performance. http://siag.nu/pen/
Re: Load balancers
That's for all the info so far. To answer some questions, hardware is a cost issue right now. It's somewhat scary that $3,200 was a reasonable price several years ago, but I suppose it could be worse. We will investigate further. The mod_rewrite solutions lack dead server detection, and that's something I'd rather not try to roll on my own, especially after seeing how well (or not, actually) existing software solutions do. But I've added it to the list. We're investigating LVS right now. It's kind of disappointing to hear that the mod_perl solution it probably not feasible. Perl solutions are always more fun to implement ;) We chose pound over pen, but we may revisit pen again. I suspect we will have similar problems with our expected load, however. Whackamole, fun name aside, does not seem to be what we need. We don't need the caching part of Squid, and I wasn't aware it did load balancing too. I'll check it out. Thanks for all the info, and please feel free to send me more, especially if there's some gem of a software load balancer out there somewhere... :) -John
Re: Load balancers
John Siracusa wrote: The mod_rewrite solutions lack dead server detection, and that's something I'd rather not try to roll on my own, especially after seeing how well (or not, actually) existing software solutions do. But I've added it to the list. ... It's kind of disappointing to hear that the mod_perl solution it probably not feasible. Perl solutions are always more fun to implement ;) The mod_rewrite option is a Perl solution. You would write some Perl code to manage the availability checks and optionally to provide a new load-balancing algorithm. It's very similar to using mod_perl for it, except you get to skip writing lots of annoying proxy code. - Perrin
Re: Load balancers
On 1/13/03 1:04 PM, Perrin Harkins wrote: John Siracusa wrote: The mod_rewrite solutions lack dead server detection, and that's something I'd rather not try to roll on my own, especially after seeing how well (or not, actually) existing software solutions do. But I've added it to the list. ... It's kind of disappointing to hear that the mod_perl solution it probably not feasible. Perl solutions are always more fun to implement ;) The mod_rewrite option is a Perl solution. You would write some Perl code to manage the availability checks and optionally to provide a new load-balancing algorithm. It's very similar to using mod_perl for it, except you get to skip writing lots of annoying proxy code. But in a full-fledged mod_perl solution, I could back out gracefully and retry another server if I happened to initially choose a dead server before my dead server detection code caught it. With the rewrite solution, I can't think of a reasonable way to keep users from ever getting passed to dead server. (Obviously I can't check the server I plan to send to on every request :) -John
RE: Load balancers
Hi John -- That's for all the info so far. To answer some questions, hardware is a cost issue right now. It's somewhat scary that $3,200 was a reasonable price several years ago, but I suppose it could be worse. We will investigate further. Actually, $3200 was a STEAL! Cisco's Local Director was in the mid five-figures at the time, IIRC. :-) It's kind of disappointing to hear that the mod_perl solution it probably not feasible. Perl solutions are always more fun to implement ;) Ah yes... Fun, for sure. I would have loved to make one, but that was just not the correct thing to do at the time. TTYL, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
Re: Load balancers
John Siracusa wrote: But in a full-fledged mod_perl solution, I could back out gracefully and retry another server if I happened to initially choose a dead server before my dead server detection code caught it. That sounds cool, but how important is it really? I'm not sure any of these solutions (including the commercial ones) do that level of seamless failover effectively. - Perrin
Re: Load balancers
On 1/13/03 1:28 PM, Perrin Harkins wrote: John Siracusa wrote: But in a full-fledged mod_perl solution, I could back out gracefully and retry another server if I happened to initially choose a dead server before my dead server detection code caught it. That sounds cool, but how important is it really? I'm not sure any of these solutions (including the commercial ones) do that level of seamless failover effectively. The five-figure ones better! ;) (they should also make me lunch) -John
RE: Load balancers
On Mon, 13 Jan 2003, Jesse Erlbaum wrote: That's for all the info so far. To answer some questions, hardware is a cost issue right now. It's somewhat scary that $3,200 was a reasonable price several years ago, but I suppose it could be worse. We will investigate further. Actually, $3200 was a STEAL! Cisco's Local Director was in the mid five-figures at the time, IIRC. :-) No, the local directors were that never that much, maybe low five-figures, like $12-15K or so. Boston.com was running on a pair of (saturated) low end LD's up until around Y2K, when we revamped the load balancing architecture and switched over to ArrowPoint (which then got gobbled up by Cisco) load distributing switches. At that time, a past-its-prime LD could be had for $3K or so, and a state-of-the-art Arrowpoint CS-150 ran around $12K (but we installed the higher capacity CS-800, which ran around $30K each -- just don't ask them to support HTTP 100% correctly... ;^) I've been out of touch with hardware load balancing equipment over the last couple of years, but if the decline in hardware costs of these things has continued, I'd guess a reasonably chunky solution can be had in the $5-7K range at this point. (Of course, you'd need to multiply that cost by two for complete redundancy...) [ Love to hear from someone with current knowledge if this is the case though...] -- Steve Reppucci [EMAIL PROTECTED] | Logical Choice Software http://logsoft.com/ | =-=-=-=-=-=-=-=-=-=- My God! What have I done? -=-=-=-=-=-=-=-=-=-=
Re: Load balancers
I've developed an embedded (Linux) load balancer solution. It's small form factor, runs on 8MB flash minimum, no hard drive, no fan so no wories about hard drive failure. It is LVS NAT (L4) based with configurable monitoring service. It load balances any port you want. I've deployed it to load balance web servers, mail server, etc. One of the busier sites is using it to load balance 6 web servers with 5M average hits a day. They've had twice that traffic and the load balancer seems to be fine. LVS says it can balance up to 24 servers. Cost $500. John Siracusa wrote: That's for all the info so far. To answer some questions, hardware is a cost issue right now. It's somewhat scary that $3,200 was a reasonable price several years ago, but I suppose it could be worse. We will investigate further. The mod_rewrite solutions lack dead server detection, and that's something I'd rather not try to roll on my own, especially after seeing how well (or not, actually) existing software solutions do. But I've added it to the list. We're investigating LVS right now. It's kind of disappointing to hear that the mod_perl solution it probably not feasible. Perl solutions are always more fun to implement ;) We chose pound over pen, but we may revisit pen again. I suspect we will have similar problems with our expected load, however. Whackamole, fun name aside, does not seem to be what we need. We don't need the caching part of Squid, and I wasn't aware it did load balancing too. I'll check it out. Thanks for all the info, and please feel free to send me more, especially if there's some gem of a software load balancer out there somewhere... :) -John
RE: Load balancers
On Mon, 13 Jan 2003, Jesse Erlbaum wrote:
Hi John --
That's for all the info so far. To answer some questions,
hardware is a cost issue right now. It's somewhat scary that
$3,200 was a reasonable price several years ago, but I
suppose it could be worse. We will investigate further.
Actually, $3200 was a STEAL! Cisco's Local Director was in the mid
five-figures at the time, IIRC. :-)
So it was that range, not $15K as I thought. Yep, I remember that we got
both of them for far far less than a Cisco, and much better support. We
had the chief developer's cell phone number, and he would help us out all
the time. At the time we were growing so fast we were almost a test case
for them ... we took them up to at least 70 million dynamic requests a
day, with stickiness required, using the adaptive balancing, with no real
problems, before moving to the Foundry Systems hardware.
- nick
Nick Tonkin {|8^)
Re: Load balancers
John Siracusa wrote: (This may seem off topic for this list, but I'll try to bring it around before the end of the message :) We've been struggling with load balancers for a while now. It seems that most experts hang at [EMAIL PROTECTED] It's also preferrable that the hw solutions will be discussed there. [...] This brings me to the mod_perl angle. Has anyone ever tried using a slimmed down mod_perl server as a load balancer? Is this feasible? Making routing decisions is obviously the easy part when using mod_perl, but would those mod_perl apache process just be too big and too slow to proxy requests efficiently? And how would they deal with detecting back-end servers that have failed? As someone has mentioned, squid is doing that (See the guide). The good thing is that it spawns the process and never quits it, so you don't have an overhead of perl startup for each request. Indeed it'll use a lot of memory. But may be toying with mod_perl 2.0 / threaded mpm will prove to be more memory efficient. Also PPerl comes to a mind. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
