Thanks again and yes, I agree - ideally at least 2 servers in each of at least 2 locations. And I'm familiar with the approach shown in your diagram for a single location, having done something similar a few years ago.
But the original point that Steven was making is, what happens if one of those location suddenly goes down completely, for whatever reason, so that all servers at that locaiton are effectively down? I think that the DNS issues I have mentioned come into play then, and I'm questioning how browsers would deal with this - I think many of them will continue to try to access the servers at the failed location. > -----Original Message----- > From: Peter Lin [mailto:[EMAIL PROTECTED] > Sent: Thursday 11 November 2004 13:12 > To: Tomcat Users List > Subject: Re: Multi-Site Clustering? (hot failover) > > > the kind of setup I've seen is to handle it with a load balancing > router and a cluster of servers. Using just one server doesn't feel > too redundant or fault tolerant to me. In the past, we had the router > direct traffic, so that if a single webserver is down, it quickly > directs it to a good server. So if the setup is like this > > ISP dns --> load balanced router (n webservers) > / | \ > / | \ > w1 w2 w3 > > if w1.mydomain.com goes down, the router should direct traffic to w2 > or w3. usually, there's some script or daemon running on each box to > make sure the webserver process is running. If not, it sends out an > email and tries to restart it. I know this approach works, but for a > single webserver in two locations, I don't know first hand. If someone > is going to the trouble of having two servers in different locations, > it makes sense to me to put atleast 2 servers in each place. The cost > is minimal and you get much better fault tolerance. > > peter > > > On Thu, 11 Nov 2004 12:56:06 -0000, Steve Kirk > <[EMAIL PROTECTED]> wrote: > > Yes that's true and I have used that feature in the past by > asking our ISP > > (registrar) to enter multiple A records in the primary DNS. > > > > However, the point of this thread is that Steven was asking > about the > > situation where one of the webservers then goes down. In this case, > > third-party ISP DNS servers will have cached both the main > and backup server > > entries, and will continue to do so for up to 48 hours (or > maybe longer in > > the case of DNSs that ignore TTL). So my understanding it > that in this > > case, browsers will potentially try to access both the > sites - including the > > one which has failed. > > > > I am basing this statement on experience from 2-3 years > ago. Maybe things > > have changed so that this is not a concern anymore? e.g. > maybe browsers are > > smarter at using multiple A records from DNS, so that if one is not > > reachable they switch to the other, or re-query the DNS, > maybe secondary > > DNSs re-query before TTL if all the A records for a domain name are > > unreachable, maybe nearly all ISPs now respect TTL? If > anyone can comment > > on these I'd be interested, because it makes Steven's original idea > > feasible. > > > > > > > > > -----Original Message----- > > > From: Filip Hanik - Dev [mailto:[EMAIL PROTECTED] > > > Sent: Thursday 11 November 2004 02:03 > > > To: Tomcat Users List > > > Subject: Re: Multi-Site Clustering? (hot failover) > > > > > > > > > if you have two TC instances running at two different IP > > > addresses and they are both up, > > > just put in DNS loadbalancing (ie, configure DNS to have both > > > IP addresses in the A records) > > > you don't need to run DNS to do this > > > > > > Filip > > > > > > ----- Original Message ----- > > > From: "Steve Kirk" <[EMAIL PROTECTED]> > > > To: "'Tomcat Users List'" <[EMAIL PROTECTED]>; > > > "'Peter Lin'" <[EMAIL PROTECTED]> > > > Sent: Wednesday, November 10, 2004 7:57 PM > > > Subject: RE: Multi-Site Clustering? (hot failover) > > > > > > > > > Thanks, interesting. But I think we're talking at crossed > > > purposes. I'm > > > not proposing running my own DNS, and I don't have an > > > existing installation. > > > Just an idea of maybe deploying two indentical TC webservers > > > at different > > > ISPs. > > > > > > What I'm saying is that if I have two identical TC servers, > > > one at ISP A and > > > one at ISP B, and my DNS is managed by one (or even both) of > > > them, then if > > > my main server fails, it doesn't matter how quick or > clever the DNS > > > management of A or B is, if users connect to my site via > > > other ISPs (C, D, > > > etc) whose DNS servers don't respect my ISP's low TTL. I know > > > that this used > > > to be a problem, I'm not sure how much it is these days. > > > > > > > -----Original Message----- > > > > From: Peter Lin [mailto:[EMAIL PROTECTED] > > > > Sent: Wednesday 10 November 2004 23:50 > > > > To: Tomcat Users List > > > > Subject: Re: Multi-Site Clustering? (hot failover) > > > > > > > > > > > > I would check the service level agreement for DNS load balancing > > > > across multiple sites. The big guys like Level3, global > > > crossing, mci > > > > have this stuff worked out so that when a DNS server > does fail, it > > > > does get routed immediately. typically that means you > actually don't > > > > handle DNS at all in your servers. > > > > > > > > the ISP handles that completely. If you were running > DNS on one of > > > > your boxes, they would have a hard time meeting the > SLA. The general > > > > rule is to set it up and then unplug the CAT5 cable and see how > > > > quickly the traffic gets routed to the other cluster. > > > > > > > > on a previous job, we did that and it was pretty > seamless. it wasn't > > > > cheap either since we had a couple of full cabinets at > two different > > > > locations. > > > > > > > > peter > > > > > > > > > > > > On Wed, 10 Nov 2004 22:52:08 -0000, Steve Kirk > > > > <[EMAIL PROTECTED]> wrote: > > > > > > > > > > OK that's roughly what I thought. But IME this does not > > > > switch things over > > > > > fast enough to count as a "hot failover". Maybe I'm not > > > > aware of a premium > > > > > service that's available, but my experience has been that > > > > DNS updates don't > > > > > propagate fast enough for this. There are often customers > > > > that cannot reach > > > > > the site after a DNS change for 24-48 hours, or even more > > > > in some cases. > > > > > > > > > > IME the problem seems to be that some secondary DNSs (e.g. > > > > those provided by > > > > > some of your end users' ISPs) don't seem to respect TTL in > > > > the DNS records, > > > > > and so they keep stale records without checking for > > > > updates, even if you > > > > > specify a short TTL. This is a problem at the best of > > > > times, let alone when > > > > > your main site has failed. It seems to me that > > > > occasionally, some DNSs also > > > > > seem to miss changes in SOA, which can be disastrous if you > > > > move your DNS to > > > > > a new ISP. > > > > > > > > > > As far as I can see, there is no way to get around these > > > > glitches because > > > > > the secondary DNSs are under the control of an ISP that you > > > > do not have a > > > > > relationship with....? Or are the problems I'm describing > > > > a thing of the > > > > > past? > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Filip Hanik - Dev [mailto:[EMAIL PROTECTED] > > > > > > Sent: Wednesday 10 November 2004 19:14 > > > > > > To: Tomcat Users List; Peter Lin > > > > > > Subject: Re: Multi-Site Clustering? (hot failover) > > > > > > > > > > > > > > > > > > which might also give you the idea, if you control your own > > > > > > DNS, you could manually switch it over to a new set of IP > > > > > > addresses when > > > > > > your old data center blows up. > > > > > > > > > > > > > > > > > > Filip > > > > > > > > > > > > ----- Original Message ----- > > > > > > From: "Peter Lin" <[EMAIL PROTECTED]> > > > > > > To: "Tomcat Users List" <[EMAIL PROTECTED]> > > > > > > Sent: Wednesday, November 10, 2004 1:08 PM > > > > > > Subject: Re: Multi-Site Clustering? (hot failover) > > > > > > > > > > > > > > > > > > normally ISP will offer multi-site load balancing using > > > > DNS. In terms > > > > > > of failover, it generally handled the same way. If > an earthquake > > > > > > swallows the first location, the second site's DNS will > > > > pick and route > > > > > > the traffic to the second cluster. > > > > > > > > > > > > I would talk to your service provider. the smaller shops > > > > don't offer > > > > > > it, so you'll have to talk to a bigger ISP. > > > > > > > > > > > > peter > > > > > > > > > > > > > > > > > > > > > > > > On Wed, 10 Nov 2004 12:52:17 -0600, Filip Hanik - Dev > > > > > > <[EMAIL PROTECTED]> wrote: > > > > > > > Even a datacenter by itself plugs in to more than one > > > > > > backbone (network provider) > > > > > > > So a datacenter itself has more than one > connection into it. > > > > > > > So what I am saying, if you want to fail over between data > > > > > > centers, that is not something you configure in tomcat, > > > > or in your own > > > > > > > network, that is something you probably arrange with the > > > > > > data centers or the network providers, cause if your data > > > > center gets > > > > > > > shattered in an earth quake, all the routers in there will > > > > > > be dead anyway. > > > > > > > > > > > > > > Filip > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > > > > > For additional commands, e-mail: > > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: > [EMAIL PROTECTED] > > > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
