Re: ISP inbound failover without BGP

[Vlade Ristevski]

I've been doing the suggestion below for many years using the IP 
addresses that Cogent gives us. All I needed to do is get  LOA from them 
and submit it to my backup ISP. I've never had an issue with my Cogent 
IP's *not* being advertised by my other ISP and I really don't think 
there is very much management overhead for the customer once this is 
setup. I have an SNMP based alerting system (Cacti) set up so I can be 
alerted if too much traffic ever "shifts" to the backup link.


The client getting their own ASN is the better way to go but you should 
be able to do the above until that comes through.



On 3/3/2014 10:20 PM, Randy Carpenter wrote:

Is there some technical reason that BGP is not an option? You could allow them to 
announce their AT&T space via you as a secondary.

-Randy

- Original Message -

This may sound like dumb question, but... I'm used to asking those.

Here's the scenario

Another ISP, say AT&T, is the primary ISP for a customer.

Customer has publicly accessible servers in their office, using the AT&T
address space.

I am the customer's secondary ISP.

Now, if AT&T link fails, I can provide the customer outbound Internet access
fairly easily.  So they can surf and get to the Internet.

What about the publicly accessible servers that have AT&T addresses, though?

One thought I had was having them use Dynamic DNS service.

Are there any other solutions, short of using BGP multihoming and having them
try to get their own ASN and IPv4 /24 block?


It looks like a few router manufacturers have devices that might work, but it
looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the
primary ISP fails, the secondary ISP address is advertised.




--
Vlad

Re: ISP inbound failover without BGP

[Sebastian Spies]

Am 04.03.2014 05:19, schrieb William Herrin:

> Reasons why dynamic DNS fails to perform as expected include:
>
> * Web browser DNS pinning can result in a customer's web browser
> holding the old IP address indefinitely.
>
> * Host-level caching of looked up names which discards the TTL.
> Remember: your desktop or laptop performs lookups against multiple
> name services, e.g. DNS, /etc/hosts, lmhosts, NIS+. DNS TTL is no
> longer in scope once the name to address map enters the generic host
> lookup mechanism. Most OSes have a fixed timeout of one sort or
> another, some old ones as long as 24 hours.

* Eyeball ISPs' DNS resolvers might tamper with TTL values.


-- SEBASTIAN SPIES lnked.in/sspies vastly.de

Re: ISP inbound failover without BGP

[Hank Nussbacher]

Have them look at radware linkproof which is designed for small shops that 
don't want to do bgp and getting their own ASN and pi address space.  Been 
around since 1999.

http://www.radware.com/Products/LinkProof/

Hank

On Mar 4, 2014 3:11 AM, Eric A Louie  wrote:
>
> This may sound like dumb question, but... I'm used to asking those.
>
> Here's the scenario
>
> Another ISP, say AT&T, is the primary ISP for a customer.
>
> Customer has publicly accessible servers in their office, using the AT&T 
> address space.
>
> I am the customer's secondary ISP.
>
> Now, if AT&T link fails, I can provide the customer outbound Internet access 
> fairly easily.  So they can surf and get to the Internet.
>
> What about the publicly accessible servers that have AT&T addresses, though?
>
> One thought I had was having them use Dynamic DNS service. 
>
> Are there any other solutions, short of using BGP multihoming and having them 
> try to get their own ASN and IPv4 /24 block?
>
> It looks like a few router manufacturers have devices that might work, but it 
> looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the 
> primary ISP fails, the secondary ISP address is advertised. 

Re: ISP inbound failover without BGP

[Jon Lewis]

On Mon, 3 Mar 2014, Justin M. Streiner wrote:

If they're not technically competent enough to handle BGP, they won't be 
technically competent enough to deal with solutions that play the short DNS 
TTL game.


As someone else mentioned in this thread - would colocating the servers be a 
workable solution for them?  Put the servers some place where the redundancy 
exists already.


My vote goes to the traditional BGP multihomed solution.  It's the right 
way to solve the problem and the easiest to manage.


If getting AT&T to do BGP and buying a BGP capable router (they don't even 
need full routes...so so anything that'll speak BGP, take a pair of 
default routes, and handle whatever their traffic level is will do) is too 
costly[1], another possible option I've not seen mentioned is VPN.  They 
could put one machine/router somewhere with decent redundancy and setup a 
VPN gateway at their office that connects to the colo'd device.


You might even offer this as a service.

Spammers have been doing this for years.  It makes moving their operations 
easier as their public facing servers get cancelled.  All they do is move 
the VPN server(s) and their systems that do all the "work" remain online 
and hidden.


[1] If only I had a dollar for every time a client said redundancy was too 
expensive to have, but when their non-redundant stuff went offline, 
they claimed to be losing millions of $ per small unit of time.


--
 Jon Lewis, MCP :)   |  I route
 |  therefore you are
_ http://www.lewis.org/~jlewis/pgp for PGP public key_

Re: ISP inbound failover without BGP

[Seth Mattinen]

On 3/3/14, 7:20 PM, Randy Carpenter wrote:
> Is there some technical reason that BGP is not an option? You could allow 
> them to announce their AT&T space via you as a secondary.


With the risk of starting holy war on how BGP works on dialup and that
providers should permit such, the OP has not specified what transport
methods is in play with AT&T which could limit such an option.

~Seth

Re: ISP inbound failover without BGP

[Justin M. Streiner]

On Mon, 3 Mar 2014, Eric A Louie wrote:

Honestly?  Because the end-customers are not technically competent 
enough to run dual-homed BGP, and we don't want to be their managed 
service providers on the IT side.  And announcing the AT&T space is fine 
until something goes wrong, and I have to troubleshoot the problem 
(Customer - "How come AT&T is down, and we're not getting inbound 
traffic to our servers?", and I discover L3 or CenturyLink isn't 
accepting my advertisement for some weird reason, but they won't fess up 
to it for a few frustrating hours)


If they're not technically competent enough to handle BGP, they won't be 
technically competent enough to deal with solutions that play the short 
DNS TTL game.


As someone else mentioned in this thread - would colocating the servers be 
a workable solution for them?  Put the servers some place where the 
redundancy exists already.


jms

Re: ISP inbound failover without BGP

[Faisal Imtiaz]

There are other elaborate solutions to accomplish this, however all of them 
would require a competent IT/Network Person to manage the network.

If we were the ISP, we would look at such a case an an opportunity, and become 
the managed service provider, for a fee (typically a premium), and provide the 
service.

As service providers, we all complain about the end-customer being a pain, but 
we often forget that it the the PITA end-customers that give us the ability to 
earn our daily bread! I think too many of us are overworked and providing 
highly under-paid services for peanuts, where we often overlook at 
opportunities to get premium value as a PITA, and not worth it...

:)

Just my personal two cents,.

Faisal Imtiaz
Snappy Internet & Telecom


- Original Message -
> From: "Eric A Louie" 
> To: "Randy Carpenter" 
> Cc: "NANOG" 
> Sent: Monday, March 3, 2014 11:49:21 PM
> Subject: Re: ISP inbound failover without BGP
> 
> Honestly?  Because the end-customers are not technically competent enough to
> run dual-homed BGP, and we don't want to be their managed service providers
> on the IT side.  And announcing the AT&T space is fine until something goes
> wrong, and I have to troubleshoot the problem (Customer - "How come AT&T is
> down, and we're not getting inbound traffic to our servers?", and I discover
> L3 or CenturyLink isn't accepting my advertisement for some weird reason,
> but they won't fess up to it for a few frustrating hours)
> 
> 
> 
> 
> 
> >____________
> > From: Randy Carpenter 
> >To: Eric A Louie 
> >Cc: NANOG 
> >Sent: Monday, March 3, 2014 7:20 PM
> >Subject: Re: ISP inbound failover without BGP
> > 
> >
> >
> >Is there some technical reason that BGP is not an option? You could allow
> >them to announce their AT&T space via you as a secondary.
> >
> >-Randy
> >
> >- Original Message -
> >> This may sound like dumb question, but... I'm used to asking those.
> >> 
> >> Here's the scenario
> >> 
> >> Another ISP, say AT&T, is the primary ISP for a customer.
> >> 
> >> Customer has publicly accessible servers in their office, using the AT&T
> >> address space.
> >> 
> >> I am the customer's secondary ISP.
> >> 
> >> Now, if AT&T link fails, I can provide the customer outbound Internet
> >> access
> >> fairly easily.  So they can surf and get to the Internet.
> >> 
> >> What about the publicly accessible servers that have AT&T addresses,
> >> though?
> >> 
> >> One thought I had was having them use Dynamic DNS service.
> >> 
> >> Are there any other solutions, short of using BGP multihoming and having
> >> them
> >> try to get their own ASN and IPv4 /24 block?
> >> 
> >> 
> >> It looks like a few router manufacturers have devices that might work, but
> >> it
> >> looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the
> >> primary ISP fails, the secondary ISP address is advertised.
> >> 
> >> 
> >
> >
> >
>

Re: ISP inbound failover without BGP

[Justin M. Streiner]

On Mon, 3 Mar 2014, Eric A Louie wrote:

Are there any other solutions, short of using BGP multihoming and 
having them try to get their own ASN and IPv4 /24 block?


For what it sounds like the customer wants to do, this really is the right 
solution.  Most everything else has some level of 'ugly hack' attached to 
it, that can cause reliability/reachability problems at inopportune times 
(as opposed to problems that happen at opportune times).


If the customer is just looking for failover capabilities, they can take 
default via BGP from both providers, prefer one over the other, and use 
some other bits to prefer one provider over for inbound connectivity. 
They would not need very beefy routers for this.


That will get you basic service redundancy.  Add a second router, and they 
can have some protection in the event of a router failure.


It all really boils down to what the customer wants and is willing to pay 
for.  If they have services that need to be reliably reachable, then using 
a time-tested design is a prudent decision.


jms

Re: ISP inbound failover without BGP

[Eric A Louie]

Honestly?  Because the end-customers are not technically competent enough to 
run dual-homed BGP, and we don't want to be their managed service providers on 
the IT side.  And announcing the AT&T space is fine until something goes wrong, 
and I have to troubleshoot the problem (Customer - "How come AT&T is down, and 
we're not getting inbound traffic to our servers?", and I discover L3 or 
CenturyLink isn't accepting my advertisement for some weird reason, but they 
won't fess up to it for a few frustrating hours)





>
> From: Randy Carpenter 
>To: Eric A Louie  
>Cc: NANOG  
>Sent: Monday, March 3, 2014 7:20 PM
>Subject: Re: ISP inbound failover without BGP
> 
>
>
>Is there some technical reason that BGP is not an option? You could allow them 
>to announce their AT&T space via you as a secondary.
>
>-Randy
>
>- Original Message -
>> This may sound like dumb question, but... I'm used to asking those.
>> 
>> Here's the scenario
>> 
>> Another ISP, say AT&T, is the primary ISP for a customer.
>> 
>> Customer has publicly accessible servers in their office, using the AT&T
>> address space.
>> 
>> I am the customer's secondary ISP.
>> 
>> Now, if AT&T link fails, I can provide the customer outbound Internet access
>> fairly easily.  So they can surf and get to the Internet.
>> 
>> What about the publicly accessible servers that have AT&T addresses, though?
>> 
>> One thought I had was having them use Dynamic DNS service.
>> 
>> Are there any other solutions, short of using BGP multihoming and having them
>> try to get their own ASN and IPv4 /24 block?
>> 
>> 
>> It looks like a few router manufacturers have devices that might work, but it
>> looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the
>> primary ISP fails, the secondary ISP address is advertised.
>> 
>> 
>
>
>

Re: ISP inbound failover without BGP

[Joe Greco]

> Depending on their business=2C using dynamic DNS providers could be a reall=
> y bad idea. If they deal only with home users who won't even know=2C it'll =
> probably work. If their customers are security-aware businesses=2C they pro=
> bably block all sites hosted with dynamic DNS systems.


Where do dynamic DNS /providers/ enter the picture?

You update *your own* DNS records.  Those could conceivably be hosted
with a dynamic DNS provider, but why not just use the API for whatever
system you currently use for DNS?


... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

Re: ISP inbound failover without BGP

[Arturo Servin]

On Mon, Mar 3, 2014 at 7:20 PM, Randy Carpenter wrote:

> Is there some technical reason that BGP is not an option? You could allow
> them to announce their AT&T space via you as a secondary.


unless it is a /26, /25 or something shorter.

Even with a /24 things may get messy.

IPv4 is coming to an end, but it is not possible for the customer to get
their own space and use BGP?

Regards,
as

Re: ISP inbound failover without BGP

[William Herrin]

On Mon, Mar 3, 2014 at 8:11 PM, Eric A Louie  wrote:
> One thought I had was having them use Dynamic DNS service.
>
> Are there any other solutions, short of using BGP multihoming
> and having them try to get their own ASN and IPv4 /24 block?

Hi Eric,

I went through this a couple years ago with continuity of operations
planning. The bottom line is: with the notable exception of
low-activity electronic mail, switching the address record in the DNS
entry will generally not work as expected. For folks serious about
reliable access to their servers, BGP isn't just the best way, it's
the only way.

Reasons why dynamic DNS fails to perform as expected include:

* Web browser DNS pinning can result in a customer's web browser
holding the old IP address indefinitely.

* Host-level caching of looked up names which discards the TTL.
Remember: your desktop or laptop performs lookups against multiple
name services, e.g. DNS, /etc/hosts, lmhosts, NIS+. DNS TTL is no
longer in scope once the name to address map enters the generic host
lookup mechanism. Most OSes have a fixed timeout of one sort or
another, some old ones as long as 24 hours.

* Custom applications with either IP addresses hardcoded into the
configuration or with getaddrinfo() called only once and the resulting
IP address held for the lifetime of the application.

* Anti-spam systems block IP addresses when receiving large quantities
of email from formerly-quiescent IP addresses. This is a problem if
your mail server sends a lot of email and suddenly switches to a new
sending IP address.

Regards,
Bill Herrin


-- 
William D. Herrin  her...@dirtside.com  b...@herrin.us
3005 Crane Dr. .. Web: 
Falls Church, VA 22042-3004

Re: ISP inbound failover without BGP

[Eric A Louie]

That's a good point Ray - thank you.




>
> From: Ray 
>To: Matthew Crocker ; Eric A Louie 
> 
>Cc: NANOG  
>Sent: Monday, March 3, 2014 6:31 PM
>Subject: RE: ISP inbound failover without BGP
> 
>
>
> 
>Depending on their business, using dynamic DNS providers could be a really bad 
>idea. If they deal only with home users who won't even know, it'll probably 
>work. If their customers are security-aware businesses, they probably block 
>all sites hosted with dynamic DNS systems.
>
>Ray
>
>
>> Subject: Re: ISP inbound failover without BGP
>> From: matt...@corp.crocker.com
>> Date: Mon, 3 Mar 2014 20:50:26 -0500
>> To: elo...@yahoo.com
>> CC: nanog@nanog.org
>> 
>> 
>> 
>> Depends on the application, 
>> 
>> SIP, VPN, SMTP, etc just setup both IPs and let the end-user application 
>> figure it out (SIP-UA register to both IPs for example)
>> 
>> HTTP/HTTPS setup a proxy server in a colo that is multi-homed to frontend 
>> the requests. Then it can load balance traffic over both IPs.
>> 
>> DNS TTL ‘tricks’ are just that, they work ‘kinda’
>> 
>> Fatpipe?   Crazy expensive IMHO but I hear they work ok.
>> 
>> -Matt
>> 
>> --
>> Matthew S. Crocker
>> President
>> Crocker Communications, Inc.
>> PO BOX 710
>> Greenfield, MA 01302-0710
>> 
>> E: matt...@crocker.com
>> P: (413) 746-2760
>> F: (413) 746-3704
>> W: http://www.crocker.com
>> 
>> 
>> 
>> On Mar 3, 2014, at 8:11 PM, Eric A Louie  wrote:
>> 
>> > This may sound like dumb question, but... I'm used to asking those.
>> > 
>> > Here's the scenario
>> > 
>> > Another ISP, say AT&T, is the primary ISP for a customer.
>> > 
>> > Customer has publicly accessible servers in their office, using the AT&T 
>> > address space.
>> > 
>> > I am the customer's secondary ISP.
>> > 
>> > Now, if AT&T link fails, I can provide the customer outbound Internet 
>> > access fairly easily.  So they can surf and get to the Internet.
>> > 
>> > What about the publicly accessible servers that have AT&T addresses, 
>> > though?
>> > 
>> > One thought I had was having them use Dynamic DNS service. 
>> > 
>> > Are there any other solutions, short of using BGP multihoming and having 
>> > them try to get their own ASN and IPv4 /24 block?
>> > 
>> > 
>> > It looks like a few router manufacturers have devices that might work, but 
>> > it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the 
>> > primary ISP fails, the secondary ISP address is advertised.
>> > 
>> 
>> 
>
>
>

Re: ISP inbound failover without BGP

[Randy Carpenter]

Is there some technical reason that BGP is not an option? You could allow them 
to announce their AT&T space via you as a secondary.

-Randy

- Original Message -
> This may sound like dumb question, but... I'm used to asking those.
> 
> Here's the scenario
> 
> Another ISP, say AT&T, is the primary ISP for a customer.
> 
> Customer has publicly accessible servers in their office, using the AT&T
> address space.
> 
> I am the customer's secondary ISP.
> 
> Now, if AT&T link fails, I can provide the customer outbound Internet access
> fairly easily.  So they can surf and get to the Internet.
> 
> What about the publicly accessible servers that have AT&T addresses, though?
> 
> One thought I had was having them use Dynamic DNS service.
> 
> Are there any other solutions, short of using BGP multihoming and having them
> try to get their own ASN and IPv4 /24 block?
> 
> 
> It looks like a few router manufacturers have devices that might work, but it
> looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the
> primary ISP fails, the secondary ISP address is advertised.
> 
> 

RE: ISP inbound failover without BGP

[Ray]

Depending on their business, using dynamic DNS providers could be a really bad 
idea. If they deal only with home users who won't even know, it'll probably 
work. If their customers are security-aware businesses, they probably block all 
sites hosted with dynamic DNS systems.

Ray

> Subject: Re: ISP inbound failover without BGP
> From: matt...@corp.crocker.com
> Date: Mon, 3 Mar 2014 20:50:26 -0500
> To: elo...@yahoo.com
> CC: nanog@nanog.org
> 
> 
> 
> Depends on the application,  
> 
> SIP, VPN, SMTP, etc just setup both IPs and let the end-user application 
> figure it out (SIP-UA register to both IPs for example)
> 
> HTTP/HTTPS setup a proxy server in a colo that is multi-homed to frontend the 
> requests. Then it can load balance traffic over both IPs.
> 
> DNS TTL ‘tricks’ are just that, they work ‘kinda’
> 
> Fatpipe?   Crazy expensive IMHO but I hear they work ok.
> 
> -Matt
> 
> --
> Matthew S. Crocker
> President
> Crocker Communications, Inc.
> PO BOX 710
> Greenfield, MA 01302-0710
> 
> E: matt...@crocker.com
> P: (413) 746-2760
> F: (413) 746-3704
> W: http://www.crocker.com
> 
> 
> 
> On Mar 3, 2014, at 8:11 PM, Eric A Louie  wrote:
> 
> > This may sound like dumb question, but... I'm used to asking those.
> > 
> > Here's the scenario
> > 
> > Another ISP, say AT&T, is the primary ISP for a customer.
> > 
> > Customer has publicly accessible servers in their office, using the AT&T 
> > address space.
> > 
> > I am the customer's secondary ISP.
> > 
> > Now, if AT&T link fails, I can provide the customer outbound Internet 
> > access fairly easily.  So they can surf and get to the Internet.
> > 
> > What about the publicly accessible servers that have AT&T addresses, though?
> > 
> > One thought I had was having them use Dynamic DNS service.  
> > 
> > Are there any other solutions, short of using BGP multihoming and having 
> > them try to get their own ASN and IPv4 /24 block?
> > 
> > 
> > It looks like a few router manufacturers have devices that might work, but 
> > it looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the 
> > primary ISP fails, the secondary ISP address is advertised.
> > 
> 
> 
  

Re: ISP inbound failover without BGP

[Matthew Crocker]

Depends on the application,  

SIP, VPN, SMTP, etc just setup both IPs and let the end-user application figure 
it out (SIP-UA register to both IPs for example)

HTTP/HTTPS setup a proxy server in a colo that is multi-homed to frontend the 
requests. Then it can load balance traffic over both IPs.

DNS TTL ‘tricks’ are just that, they work ‘kinda’

Fatpipe?   Crazy expensive IMHO but I hear they work ok.

-Matt

--
Matthew S. Crocker
President
Crocker Communications, Inc.
PO BOX 710
Greenfield, MA 01302-0710

E: matt...@crocker.com
P: (413) 746-2760
F: (413) 746-3704
W: http://www.crocker.com



On Mar 3, 2014, at 8:11 PM, Eric A Louie  wrote:

> This may sound like dumb question, but... I'm used to asking those.
> 
> Here's the scenario
> 
> Another ISP, say AT&T, is the primary ISP for a customer.
> 
> Customer has publicly accessible servers in their office, using the AT&T 
> address space.
> 
> I am the customer's secondary ISP.
> 
> Now, if AT&T link fails, I can provide the customer outbound Internet access 
> fairly easily.  So they can surf and get to the Internet.
> 
> What about the publicly accessible servers that have AT&T addresses, though?
> 
> One thought I had was having them use Dynamic DNS service.  
> 
> Are there any other solutions, short of using BGP multihoming and having them 
> try to get their own ASN and IPv4 /24 block?
> 
> 
> It looks like a few router manufacturers have devices that might work, but it 
> looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the 
> primary ISP fails, the secondary ISP address is advertised.
> 

Re: ISP inbound failover without BGP

[Joe Greco]

> This may sound like dumb question, but... I'm used to asking those.=0A=0AHe=
> re's the scenario=0A=0AAnother ISP, say AT&T, is the primary ISP for a cust=
> omer.=0A=0ACustomer has publicly accessible servers in their office, using =
> the AT&T address space.=0A=0AI am the customer's secondary ISP.=0A=0ANow, i=
> f AT&T link fails, I can provide the customer outbound Internet access fair=
> ly easily.=A0 So they can surf and get to the Internet.=0A=0AWhat about the=
>  publicly accessible servers that have AT&T addresses, though?=0A=0AOne tho=
> ught I had was having them use Dynamic DNS service.=A0 =0A=0AAre there any =
> other solutions, short of using BGP multihoming and having them try to get =
> their own ASN and IPv4 /24 block?=0A=0A=0AIt looks like a few router manufa=
> cturers have devices that might work, but it looks like a short DNS TTL (or=
>  Dynamic DNS) needs to be set so when the primary ISP fails, the secondary =
> ISP address is advertised.

The usual solution is to get the public servers stuck in a colo that's
multihomed.

Most of the other solutions tend to be a bit dodgy.  If your gear is
sufficiently competent, you can hack up a solution with multiple 
addresses for each of the servers (one on each ISP) and then use a
short TTL to fail over, but this has more of "fail" than "fail over"
about it, because there are a bunch of issues that typically result.

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.

ISP inbound failover without BGP

[Eric A Louie]

This may sound like dumb question, but... I'm used to asking those.

Here's the scenario

Another ISP, say AT&T, is the primary ISP for a customer.

Customer has publicly accessible servers in their office, using the AT&T 
address space.

I am the customer's secondary ISP.

Now, if AT&T link fails, I can provide the customer outbound Internet access 
fairly easily.  So they can surf and get to the Internet.

What about the publicly accessible servers that have AT&T addresses, though?

One thought I had was having them use Dynamic DNS service.  

Are there any other solutions, short of using BGP multihoming and having them 
try to get their own ASN and IPv4 /24 block?


It looks like a few router manufacturers have devices that might work, but it 
looks like a short DNS TTL (or Dynamic DNS) needs to be set so when the primary 
ISP fails, the secondary ISP address is advertised.