Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew, Andrew Miehs wrote: If you need two devices load balancers to deal with your current load, then you already have a problem. I was thinking from a failover standpoint, not so much a too much load for a single piece of hardware stance. These things should be able to handle more traffic that you can get through your pipe. We run our BigIPs Active/Standby - and when the one BigIP dies, the other takes over the 'VIPs' - virtual ips - from the other box. The one IP address points to a floating address which is shared by the load balancer. That makes sense, and is more convenient than R-R DNS. My only experience with BigIP was that they were used at an unnamed registrar/SSL-cert provider as a contractor for about two years. The ops folks took care of all of that, leaving us to work on the actual software. HA solutions work the same way, moving a floating addresses between the multiple boxes... Sounds good to me. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF8BgQ9CaO5/Lv0PARAo0qAKCsRdJwWBJxKuVYwehoYLvzIzuL3ACdGBaw 1cjg4vXG+GI3u57liSf5T1g= =UyTV -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
Hi all: I'd posted sometime ago seeking help for a particular requirement. Rainer Jung replied to my post. The thread is here http://thread.gmane.org/gmane.comp.jakarta.tomcat.user/144823 Requirement: 1. Host an application on two tomcat instances. 2. Enable load balancing between these two Tomcat instances. 3. If one Tomcat instance goes down then the other should take over. 4. If the second goes down, then the first should be retried. This should happen even if the first had failed some time ago. 5. After both the Tomcat instances are retried a number of times, fall back to yet another Tomcat. This third tomcat could do some special processing. Solution: mod_jk gives us this out of the box. For load balancing, add the first two Tomcat instances worker.router.balance_workers=worker1,worker2 For fail over between the two instances, do nothing extra For making only the first one work, and to use the second instance only for fail over, mark the second instance as disabled. worker.worker2.activation=disabled For ensuring that mod_jk tries both the Tomcat instances a specified number of times before moving to the third Tomcat instance. ensure that the first two Tomcat instances have the same distance count, while the Third tomcat instance has a higher distance count. worker.worker1.distance=0 worker.worker2.distance=0 worker.worker3.distance=1 I've attached the worker.properties file below for reference. == worker.properties = # The advanced router LB worker worker.list=router # Define a worker using ajp13 worker.worker1.port=8009 worker.worker1.recovery_options=0 worker.worker1.host=192.168.11.26 worker.worker1.type=ajp13 worker.worker1.lbfactor=1 worker.worker1.retries=3 worker.worker1.fail_on_status=600 worker.worker1.distance=0 worker.worker1.mount=/mondrian/* # Define prefered failover node for worker1 worker.worker1.redirect=worker2 # Define another worker using ajp13 worker.worker2.port=8009 worker.worker2.recovery_options=0 worker.worker2.host=192.168.11.25 worker.worker2.type=ajp13 worker.worker2.lbfactor=1 worker.worker2.retries=3 worker.worker2.fail_on_status=600 worker.worker2.distance=0 worker.worker2.mount=/mondrian/* # Disable worker2 for all requests except failover. Comment out the following to enable the 2 node load-balancing # worker.worker2.activation=disabled # Define the last resort worker using ajp13 worker.worker3.port=8009 worker.worker3.recovery_options=0 worker.worker3.host=192.168.11.24 worker.worker3.type=ajp13 worker.worker3.lbfactor=1 worker.worker3.retries=3 worker.worker3.fail_on_status=600 worker.worker1.distance=1 worker.worker3.mount=/mondrian/* # Define the LB worker worker.router.type=lb worker.router.sticky_session=0 #worker.router.sticky_session_force=0 worker.router.balance_workers=worker1,worker2 worker.list=jkstatus # Define a 'jkstatus' worker using status worker.jkstatus.type=status - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This will work if you are only using 1 Apache HTTPD server Regards Andrew On 07/03/2007, at 11:27 AM, Sriram Narayanan wrote: I'd posted sometime ago seeking help for a particular requirement. Rainer Jung replied to my post. The thread is here http://thread.gmane.org/gmane.comp.jakarta.tomcat.user/144823 Requirement: 1. Host an application on two tomcat instances. 2. Enable load balancing between these two Tomcat instances. 3. If one Tomcat instance goes down then the other should take over. 4. If the second goes down, then the first should be retried. This should happen even if the first had failed some time ago. 5. After both the Tomcat instances are retried a number of times, fall back to yet another Tomcat. This third tomcat could do some special processing. Solution: mod_jk gives us this out of the box. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Darwin) iD8DBQFF7pTSW126qUNSzvURAjE7AJ4z/2hNk4QqdFirX0liLH0YBDJWVwCfZRwh bY/ufculwvdURDGYguWx+Gw= =euml -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
On 3/7/07, Andrew Miehs [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This will work if you are only using 1 Apache HTTPD server Are you referring to the following ? Two httpd servers with mod_jk pointing to the same two tomcat instances. httpdA and mod_jkA have a reference to TomcatA and TomcatB httpdB and mod_jkB have a reference to TomcatA and TomcatB What would be the complications in such a scenario ? I can think of situation where mod_jkA does not know of mod_jkB's status and vice-versa. In such a case, let's assume that TomcatA is down and that mod_jkB learns of this. And that mod_jk A has not yet contacted TomcatA. For long running requests, it would definitely be a problem to learn of TomcatA being unable to serve requests quite some time after sending a request to it. If the processing times on tomcat are low, would it be acceptable for mod_jkA to learn of the TomcatA instance being down, updating it's internal list with this info, and then redirecting the request to tomcatB. This is all that I could think of. If there's something else that we should be knowing, please do write back. Thanks in advance. -- Sriram - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/03/2007, at 12:58 PM, Sriram Narayanan wrote: On 3/7/07, Andrew Miehs [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This will work if you are only using 1 Apache HTTPD server Are you referring to the following ? Two httpd servers with mod_jk pointing to the same two tomcat instances. httpdA and mod_jkA have a reference to TomcatA and TomcatB httpdB and mod_jkB have a reference to TomcatA and TomcatB What would be the complications in such a scenario ? I can think of situation where mod_jkA does not know of mod_jkB's status and vice-versa. Yep - this is the case I am referring to. You have moved the problem of load balancing Tomcat to load balancing HTTPD. How do you want to load balance httpd? Don't forget, you will need something to deal with session persistence. If the Apache HTTPD and tomcat are running on the same machine, you will probably find it easier to do a 1-1 mapping httpd - tomcat than the cross over setup that you are currently envisaging. Regards Andrew -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Darwin) iD8DBQFF7rWrW126qUNSzvURAu8qAJ9a1lR//+cKOr9xYa5q4byFn2IltQCcCU5Q esRNcj1ucsQNwA3K+XLjrGE= =4Tlr -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew, Andrew Miehs wrote: This will work if you are only using 1 Apache HTTPD server Really? It looks like it would work to me. Sure, the separate mod_jks don't know each other's status, but it doesn't matter as they will quickly find out the status of each Tomcat instance pretty quickly. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF7tka9CaO5/Lv0PARAqWPAKCa5U7KcTmUqRB26TvWppEZfvY75gCfe+l3 qLx+XBYTcKrDRCHdzghPQ0g= =KPJ6 -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Christopher, Balancing the 2 Tomcats behind one Apache (with sticky sessions) works. Now you add a second Apache HTTPD. How do you choose which one of these gets used? You now have the original problem all over again... How do you load between the two web servers? Cheers Andrew On 07/03/2007, at 4:24 PM, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew, Andrew Miehs wrote: This will work if you are only using 1 Apache HTTPD server Really? It looks like it would work to me. Sure, the separate mod_jks don't know each other's status, but it doesn't matter as they will quickly find out the status of each Tomcat instance pretty quickly. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Darwin) iD8DBQFF7u9CW126qUNSzvURAh5+AKCEcI3uKAisPAKhRUuTEMXSHSWzqACffxxx 5YZuzPz+e44Lq4EI3EV+SX8= =du2M -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew, Andrew Miehs wrote: Balancing the 2 Tomcats behind one Apache (with sticky sessions) works. Now you add a second Apache HTTPD. How do you choose which one of these gets used? You now have the original problem all over again... How do you load between the two web servers? Perhaps round-robin DNS? That's how I would do it, unless I wanted to buy a real load balancer like a BigIP. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF7vW79CaO5/Lv0PARAp36AKC/dEjFHuhANbnFdDRf9RdnY3Ew+gCgn0hT j2/q1J9dDj1XWgl2DEtj/7w= =V9Ll -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
On 3/7/07, Christopher Schultz [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew, Andrew Miehs wrote: Balancing the 2 Tomcats behind one Apache (with sticky sessions) works. Now you add a second Apache HTTPD. How do you choose which one of these gets used? You now have the original problem all over again... How do you load between the two web servers? Perhaps round-robin DNS? That's how I would do it, unless I wanted to buy a real load balancer like a BigIP. Ok, round-robin dns will work. But it will probably work with pure tomcats too, wouldn't it? If you round-robin the load between two httpds, why dont you do the same between two tomcats? Leon - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF7vW79CaO5/Lv0PARAp36AKC/dEjFHuhANbnFdDRf9RdnY3Ew+gCgn0hT j2/q1J9dDj1XWgl2DEtj/7w= =V9Ll -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Leon, Leon Rosenberg wrote: On 3/7/07, Christopher Schultz [EMAIL PROTECTED] wrote: Perhaps round-robin DNS? That's how I would do it, unless I wanted to buy a real load balancer like a BigIP. Ok, round-robin dns will work. But it will probably work with pure tomcats too, wouldn't it? If you round-robin the load between two httpds, why dont you do the same between two tomcats? I suppose we're getting into a semi-philosophical discussion, here, about where load balancing belongs. In my opinion, Tomcat is the resource that needs to be load-balanced, since it's doing most of the work. You want to protect individual servers from being crushed under the weight of too much traffic. With no (formal) load balancer and just round-robin DNS, you do not achieve your goal. By the luck of the draw, all your customers could get a single server in your server bank. Or, by similar coincidence, only heavy users might single out a particular server. This is not load balancing: it's random request (or user, depending on DNS attitude of the client) distribution. With Apache httpd out front, mod_jk can monitor the status of the connected Tomcats and choose the best one for any given request. Apache httpd is not doing too much work: mostly just copying data between buffers. This does in fact load balance the two Tomcats. The question was what to do at this point?. Well, if Apache httpd goes down, you're completely hosed. So, a solution would be to setup another Apache httpd in parallel. This way, when one Apache goes down, the other is available (though not necessarily reachable when using round-robin DNS). The point is that you are not 100% down. Using round-robin DNS is not load balancing, but it will increase the robustness of your deployment. AFAICT, this robustness is not possible using Tomcats only. One needs to have something like mod_jk working to load balance the app servers. I'd be glad to hear that Tomcat can offload work to another server in a cluster, but I have heard nothing about such a feature, so I assume that it does not exist. Load balancing pretty much always comes down to either: 1. A single point of failure (Apache httpd, BigIP, or whatever). or 2. Non-100% reachability (for instance, due to one of the round-robin'd web servers going down). It seems to me that the most robust deployment for a webapp is: Random request distribution + Apache httpd + lb'd Tomcat - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF7xwK9CaO5/Lv0PARAn4SAKCQyaWTBmiDrcGt0YxR5WL+C6TyJgCfcEUi DZ26oLSl/xT0oRBg6Y+rmDA= =x1/N -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
On 3/7/07, Christopher Schultz [EMAIL PROTECTED] wrote: Load balancing pretty much always comes down to either: 1. A single point of failure (Apache httpd, BigIP, or whatever). or 2. Non-100% reachability (for instance, due to one of the round-robin'd web servers going down). At which point you use `ifconfig` to add the failed IP to the working httpd box and it's back to 100% :-) -- Hassan Schroeder [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
On 3/7/07, Christopher Schultz [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Leon, In my opinion, Tomcat is the resource that needs to be load-balanced, since it's doing most of the work. You want to protect individual servers from being crushed under the weight of too much traffic. actually usually the resources behind tomcat doing the most work. In a good scaling system you have a factor 1:5 or something... 1 DB - 5 APP servers - 25 tomcats... to give an example. But ok, its not related to the example. With no (formal) load balancer and just round-robin DNS, you do not achieve your goal. By the luck of the draw, all your customers could get a single server in your server bank. Or, by similar coincidence, only heavy users might single out a particular server. This is not load balancing: it's random request (or user, depending on DNS attitude of the client) distribution. Right, but this is something the tomcat could easily handle itself. By a filter or a valve. Would mean one more request per client. With Apache httpd out front, mod_jk can monitor the status of the connected Tomcats and choose the best one for any given request. Apache httpd is not doing too much work: mostly just copying data between buffers. This does in fact load balance the two Tomcats. The question was what to do at this point?. Well, if Apache httpd goes down, you're completely hosed. So, a solution would be to setup another Apache httpd in parallel. This way, when one Apache goes down, the other is available (though not necessarily reachable when using round-robin DNS). The point is that you are not 100% down. Using round-robin DNS is not load balancing, but it will increase the robustness of your deployment. AFAICT, this robustness is not possible using Tomcats only. One needs to have something like mod_jk working to load balance the app servers. I'd be glad to hear that Tomcat can offload work to another server in a cluster, but I have heard nothing about such a feature, so I assume that it does not exist. Maybe, but its not hard to implement, especially if you are implementing it for one app knowing exactly the load on this app and the use-cases. Load balancing pretty much always comes down to either: 1. A single point of failure (Apache httpd, BigIP, or whatever). Loadbalancers usually come in pairs :-) or 2. Non-100% reachability (for instance, due to one of the round-robin'd web servers going down). It seems to me that the most robust deployment for a webapp is: Random request distribution + Apache httpd + lb'd Tomcat paired firewalls + paired loadbalancers + tomcat cluster. Performs much better as the above :-) Try it out, give it a chance. Leon - -chris - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 :-) We just had this discussion last week on the Debian ISP mailing list. Round Robin DNS is a nasty fix to this problem, and isn't guaranteed to work correctly. Either a real load balancer (like a BigIP) or some form of Linux HA are the only real ways of dealing with this. Cheers Andrew On 07/03/2007, at 6:26 PM, Christopher Schultz wrote: Andrew Miehs wrote: Balancing the 2 Tomcats behind one Apache (with sticky sessions) works. Now you add a second Apache HTTPD. How do you choose which one of these gets used? You now have the original problem all over again... How do you load between the two web servers? Perhaps round-robin DNS? That's how I would do it, unless I wanted to buy a real load balancer like a BigIP. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Darwin) iD8DBQFF70R+W126qUNSzvURAvenAJ9Z53iM+L5wzca7TbMx86hyuFzXnQCfQJSy kzvxgXrEVlzWcgJyuJA2uAo= =2cYH -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/03/2007, at 7:47 PM, Leon Rosenberg wrote: On 3/7/07, Christopher Schultz [EMAIL PROTECTED] wrote: Perhaps round-robin DNS? That's how I would do it, unless I wanted to buy a real load balancer like a BigIP. Ok, round-robin dns will work. But it will probably work with pure tomcats too, wouldn't it? If you round-robin the load between two httpds, why dont you do the same between two tomcats? You may want to have a look at this http://homepages.tesco.net/J.deBoynePollard/FGA/dns-round-robin-is- useless.html and http://en.wikipedia.org/wiki/Round_robin_DNS Some desktop clients may even try alternate addresses after a connection time out of 30-45 seconds. This behavior is unfortunately not specified - and could be changed at any time. I don't know if I would want to define my failover via a not specified mechanism. Regards Andrew -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Darwin) iD8DBQFF70WDW126qUNSzvURAvDQAJ9nPySRDp3cDs9BSqHb+A3t6dAEmgCePuTu 025lDxVLvPXpX/GYbSC22Gg= =0FVL -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Leon, Leon Rosenberg wrote: On 3/7/07, Christopher Schultz [EMAIL PROTECTED] wrote: Load balancing pretty much always comes down to either: 1. A single point of failure (Apache httpd, BigIP, or whatever). Loadbalancers usually come in pairs :-) Er, you can buy a single load balancer off the shelf. One input port, many output ports (virtually, that is). It's not uncommon to see a single device acting as a load balancer. Load-balanced /resources/, on the other hand, generally come in multiples (otherwise, what's the point?). If you had a pair of load balancers, how would you pick which one handles the request? ... R-R DNS, anyone? It seems to me that the most robust deployment for a webapp is: Random request distribution + Apache httpd + lb'd Tomcat paired firewalls + paired loadbalancers + tomcat cluster. Performs much better as the above :-) Try it out, give it a chance. But the request has to come from somewhere and go to a single device. If you have pairs of things, you have to divide the traffic, which brings me back to R-R DNS. Otherwise, you have a set of hardware that never gets used. There's always the possibility of redirecting to another machine name, such as rack0.foo.com versus rack1.foo.com, each of which point to a particular piece of load-balancing hardware (or logical equivalent such as a firewall /in front/ of a load balancer). I'm not sure how your better layout is any different than mine, except that you've replaced Apache-httpd-based-load-balancing with what looks like appliance-based-load-balancing and put firewalls out front (which is logically insignificant). - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF71gr9CaO5/Lv0PARAitgAJ49c8f9YTEclevh6P54J3dIJmUbhQCeJgnT kZglVgOgx96gJ6hogCbjPtw= =lHsr -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew, Andrew Miehs wrote: We just had this discussion last week on the Debian ISP mailing list. Round Robin DNS is a nasty fix to this problem, and isn't guaranteed to work correctly. Certainly not. But it's one way to /actually/ divide requests between physical hardware devices. Either a real load balancer (like a BigIP) or some form of Linux HA are the only real ways of dealing with this. I totally agree. A single BigIP is a single point of failure, though. R-R DNS with multiple BigIPs is better than single IP - single BigIP, no? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF71i39CaO5/Lv0PARAlhlAKCnL8RFO0HCyFZwEGd2nLemNnZQPwCfYVM6 OEx8KwCo062JcXgYbjeDoe8= =CxuF -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew, Andrew Miehs wrote: You may want to have a look at this http://homepages.tesco.net/J.deBoynePollard/FGA/dns-round-robin-is-useless.html No offense taken, but I wasn't (as the author of this piece asserts) claiming that R-R DNS is an effective load balancer. In fact, I have said quite the opposite: load balancing is best done by a component which actually understands loads: 1. mod_jk (right? I've never done it) 2. BigIP or some other hardware load balancer The /only/ reason, IMO, to ever use R-R DNS is to avoid single points of failure. The author claims that there is no reason, ever (ever ever ever) to use R-R DNS. I respectfully disagree. If your client attempts to lookup www.foo.com, it resolves to a single IP address, which points to a single piece of hardware in your data center, you might be screwed by: 1. Faulty wiring that happens to go bad at an inconvenient time. 2. Faulty hardware device (fw, lb, switch, anything) that dies. 3. Network or power going down (which is a stretch, since data centers are pretty good at keeping the lights on) R-R DNS allows you to /partially/ weather this storm by diverting an unpredictable amount of traffic to another hardware device (possibly in another data center, which gets you around all of the above). Sure, some of your clients won't be able to connect. But, not /all/ of them will be denied service. This author claims that the following foundations are flawed reasons for using R-R DNS. Note that I do not claim a single one of them: 1. Shuffling resource records affects client connection behavior. (I don't care... my only assumption is that not every client can possibly conspire to choose the same IP address every single time). 2. Shuffling resource records provides even or predictable distribution (I don't care... it's enough that not all requests go to the same place. The distribution is is irrelevant, as long as not every single request goes to the same IP every time). The whole point is that you have to suffer 100% loss of your frontend hardware in order to shut off 100% of your users. This is true no matter how many points of failure you have... it's just that 1 point if failure means that only one device has to go. If you have a dozen lb's (or fw's with lb's, as Leon suggests), then losing 1 device loses you a completely unpredictable 1/12th of your users. If R-R DNS works perfectly (which it doesn't), then you still lose 1/12th of all requests. But the worst case simply can't be that 1 of 12 servers going down results in 100% request loss. I am unaware of any other strategy which allows you to lose a primary piece of hardware such as a load balancer and still be able to limp along with at least /some/ requests going through. I'm open to suggestions. (And SRV doesn't count, since not a single web browser supports them). - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF712f9CaO5/Lv0PARAs02AJ0b1mJWg+bRXidicTpQH5NNYaDg3QCdEpDX hUtDnuLQH8k2KT5mOaWYWqA= =3BXP -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/03/2007, at 1:28 AM, Christopher Schultz wrote: Either a real load balancer (like a BigIP) or some form of Linux HA are the only real ways of dealing with this. I totally agree. A single BigIP is a single point of failure, though. R-R DNS with multiple BigIPs is better than single IP - single BigIP, no? IMHO, not really. If you need two devices load balancers to deal with your current load, then you already have a problem. We run our BigIPs Active/Standby - and when the one BigIP dies, the other takes over the 'VIPs' - virtual ips - from the other box. The one IP address points to a floating address which is shared by the load balancer. HA solutions work the same way, moving a floating addresses between the multiple boxes... Andrew -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Darwin) iD8DBQFF747wW126qUNSzvURAoSqAJ9O+TKKJ85J2GtU1PW2T6HpYI/dpwCdGciA HAZLfdqboHY8aCI+EwEVdqY= =/Abo -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Another howto: Load Balancer + fail over (two active Tomcats, one backup Tomcat)
On 3/8/07, Christopher Schultz [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Leon, Leon Rosenberg wrote: On 3/7/07, Christopher Schultz [EMAIL PROTECTED] wrote: Load balancing pretty much always comes down to either: 1. A single point of failure (Apache httpd, BigIP, or whatever). Loadbalancers usually come in pairs :-) Er, you can buy a single load balancer off the shelf. One input port, many output ports (virtually, that is). It's not uncommon to see a single device acting as a load balancer. Load-balanced /resources/, on the other hand, generally come in multiples (otherwise, what's the point?). If you had a pair of load balancers, how would you pick which one handles the request? ... R-R DNS, anyone? one is active one is standby. Same applies to the firewall and the switches. It seems to me that the most robust deployment for a webapp is: Random request distribution + Apache httpd + lb'd Tomcat paired firewalls + paired loadbalancers + tomcat cluster. Performs much better as the above :-) Try it out, give it a chance. But the request has to come from somewhere and go to a single device. If you have pairs of things, you have to divide the traffic, which brings me back to R-R DNS. Otherwise, you have a set of hardware that never gets used. andrew already explained how this works with a pair of lbs. I'm not sure how your better layout is any different than mine, except that you've replaced Apache-httpd-based-load-balancing with what looks like appliance-based-load-balancing and put firewalls out front (which is logically insignificant). much more availability? Ideally, in a stateless or session-lbed application the user will never be affected if a machine dies :-) And yes the firewalls are insignificant. But nowerdays, with all that bot-nets and stuff, you really want them to handle the dos :-) - -chris regards Leon - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]