Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Jason Canady]

I completely concur.  We spread our uplinks across separate boxes and we have 
/29 allocations.  Get the best of all worlds. But if I only had one provider, 
I'd want to have multiple BGP sessions for this reason.  

> On Oct 17, 2016, at 08:30, Mike Hammett <na...@ics-il.net> wrote:
> 
> It really seems like it's a grave oversight to *NOT* support multiple BGP 
> sessions. I drop to two routers for that same reason, I can do maintenance on 
> one, while the other carries traffic. 
> 
> 
> 
> 
> - 
> Mike Hammett 
> Intelligent Computing Solutions 
> http://www.ics-il.com 
> 
> Midwest-IX 
> http://www.midwest-ix.com 
> 
> - Original Message -
> 
> From: "Mike Poublon" <mpoub...@secantnet.net> 
> To: "rar" <r...@syssrc.com>, nanog@nanog.org 
> Sent: Thursday, October 13, 2016 2:04:29 PM 
> Subject: Re: Two BGP peering sessions on single Comcast Fiber Connection? 
> 
> I started a thread around the same topic back on 10/16 of 2014. A 
> Comcast engineer (who ultimately spoke to the national product manager) 
> came back after discussing and said the same thing "We don't support 
> that". I got a slightly longer explanation of: 
> 
>  
> 
> In a nutshell, when we design a product we do it to accommodate the most 
> typical customer cases. 
> Given that the design includes a single fiber path and thus the fiber 
> path and device that terminates on either end each are a single point of 
> failure, adding extra BGP sessions doesn’t seem to add value in the 
> typical failure scenarios. In order to achieve the simplest and most 
> scalable solution to address the market, we rely on narrowing the 
> possible combinations of parameters. 
> 
>  
> 
> I explained to them that their interpretation prevents me from being 
> able to do concurrent maintenance on my side (single router 
> reboot/upgrade, etc). Never got anywhere with it though. 
> 
> I'm still interested in having this set up, but have given up on it ever 
> really coming to reality. Luckily ALL of my other providers were more 
> than happy to set up an extra session. 
> 
> If anyone from Comcast is listening, there is customer demand for this. 
> It's not about making it better for Comcast, it's about allowing 
> customers to have more flexibility. 
> 
> Mike Poublon 
> 
> /Senior Datacenter Network Engineer/ 
> 
> *Secant Technologies* 
> 
> 6395 Technology Ave. Suite A 
> 
> Kalamazoo, MI 49009 
> 
>> On 10/13/2016 1:48 PM, rar wrote: 
>> After a many month wait, we were ready to turn up our BGP peering sessions 
>> on a new Comcast fiber connection. 
>> 
>> With our other providers (Level 3 and Verizon) we have edge routers that 
>> directly connect between the provider's on premise connection and our 
>> primary and a backup core routers. Each core router has a multihop BGP 
>> session with the provider's BGP router. The goal is to keep the single BGP 
>> router from being a single point of failure. 
>> 
>> Comcast said they could not support two separate BGP peering sessions on the 
>> same circuit. Does anyone have any counter examples? We used to have this 
>> setup with Comcast 5+ years ago, but now they say they can't support it. 
>> 
>> 
>> Bob Roswell 
>> brosw...@syssrc.com<mailto:brosw...@syssrc.com> 
>> 410-771-5544 ext 4336 
>> 
>> Computer Museum Highlights<http://museum.syssrc.com/>
> 
> 

Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Mike Hammett]

It really seems like it's a grave oversight to *NOT* support multiple BGP 
sessions. I drop to two routers for that same reason, I can do maintenance on 
one, while the other carries traffic. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

- Original Message -

From: "Mike Poublon" <mpoub...@secantnet.net> 
To: "rar" <r...@syssrc.com>, nanog@nanog.org 
Sent: Thursday, October 13, 2016 2:04:29 PM 
Subject: Re: Two BGP peering sessions on single Comcast Fiber Connection? 

I started a thread around the same topic back on 10/16 of 2014. A 
Comcast engineer (who ultimately spoke to the national product manager) 
came back after discussing and said the same thing "We don't support 
that". I got a slightly longer explanation of: 

 

In a nutshell, when we design a product we do it to accommodate the most 
typical customer cases. 
Given that the design includes a single fiber path and thus the fiber 
path and device that terminates on either end each are a single point of 
failure, adding extra BGP sessions doesn’t seem to add value in the 
typical failure scenarios. In order to achieve the simplest and most 
scalable solution to address the market, we rely on narrowing the 
possible combinations of parameters. 

 

I explained to them that their interpretation prevents me from being 
able to do concurrent maintenance on my side (single router 
reboot/upgrade, etc). Never got anywhere with it though. 

I'm still interested in having this set up, but have given up on it ever 
really coming to reality. Luckily ALL of my other providers were more 
than happy to set up an extra session. 

If anyone from Comcast is listening, there is customer demand for this. 
It's not about making it better for Comcast, it's about allowing 
customers to have more flexibility. 

Mike Poublon 

/Senior Datacenter Network Engineer/ 

*Secant Technologies* 

6395 Technology Ave. Suite A 

Kalamazoo, MI 49009 

On 10/13/2016 1:48 PM, rar wrote: 
> After a many month wait, we were ready to turn up our BGP peering sessions on 
> a new Comcast fiber connection. 
> 
> With our other providers (Level 3 and Verizon) we have edge routers that 
> directly connect between the provider's on premise connection and our primary 
> and a backup core routers. Each core router has a multihop BGP session with 
> the provider's BGP router. The goal is to keep the single BGP router from 
> being a single point of failure. 
> 
> Comcast said they could not support two separate BGP peering sessions on the 
> same circuit. Does anyone have any counter examples? We used to have this 
> setup with Comcast 5+ years ago, but now they say they can't support it. 
> 
> 
> Bob Roswell 
> brosw...@syssrc.com<mailto:brosw...@syssrc.com> 
> 410-771-5544 ext 4336 
> 
> Computer Museum Highlights<http://museum.syssrc.com/> 
> 

Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Kraig Beahn]

Steering clear of the failure domain conversation, if its of any benefit -
we can at least confirm that Comcast is willing to establish /29's for
multiple BGP connections at 56 Marietta/ATL.

These circuits are written on true wholesale/transit IP service contracts,
which may be the difference.

In our experience the Comcast Enterprise/Business groups have rather rigid
circuit provisioning profiles, and even if you are able to talk an engineer
into building a customer's configuration outside of their normal "scope",
it usually comes back to haunt you at some point in the future, even if
years later.

Will send a link to the Comcast enterprise ip transit profiles separately,
for reference, in the event you were not provided such previously...Or if
Comcast wholesale is on the list, of course feel free to chime in too!





On Fri, Oct 14, 2016, 1:49 PM Bill Blackford  wrote:

> It comes down to sizing your failure domain. Any single upstream Transit
> alone means the failure domain is the whole site (making assumptions about
> your topology). As mentioned earlier, any single point of failure doesn't
> reduce your failure footprint and gives little in terms of redundancy. Now
> if you point that second router to a second provider, now you've reduced
> the size of your failure domain to a single router/Transit, not the whole
> site.
>
> -b
>
>
> On Fri, Oct 14, 2016 at 10:34 AM, Paul S.  wrote:
>
> > +1, could not have said it better.
> >
> >
> > On 10/15/2016 01:47 AM, Leo Bicknell wrote:
> >
> >> In a message written on Thu, Oct 13, 2016 at 05:48:18PM +, rar
> wrote:
> >>
> >>> The goal is to keep the single BGP router from being a single point of
> >>> failure.
> >>>
> >> I don't really understand the failure analysis / uptime calculation.
> >>
> >> There is one router on the Comcast side, which is a single point of
> >> failure.
> >>
> >> There is one circuit to your prem, which is a single point of failure.
> >>
> >> To connect two routers on your end you must terminate the circuit
> >> in a switch, which is a single point of failure.
> >>
> >> And yet, in the face of all that somehow running two routers with
> >> two BGP sessions on your end increases your uptime?
> >>
> >> The only way that would even remotely make sense is if the routers
> >> in question were horribly broken / mismanaged so (had to be?) reboot(ed)
> >> on a regular basis.  However if uptime is so important using gear
> >> with that property makes no sense!
> >>
> >> I'm pretty sure without actually doing the math that you'll be more
> >> reliable with a single quality router (elminiation of complexity),
> >> and that if you really need maximum uptime that you had better get
> >> a second circuit, on a diverse path, into a different router probably
> >> from a different carrier.
> >>
> >>
> >
>
>
> --
> Bill Blackford
>
> Logged into reality and abusing my sudo privileges.
>
--

Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Bill Blackford]

It comes down to sizing your failure domain. Any single upstream Transit
alone means the failure domain is the whole site (making assumptions about
your topology). As mentioned earlier, any single point of failure doesn't
reduce your failure footprint and gives little in terms of redundancy. Now
if you point that second router to a second provider, now you've reduced
the size of your failure domain to a single router/Transit, not the whole
site.

-b


On Fri, Oct 14, 2016 at 10:34 AM, Paul S.  wrote:

> +1, could not have said it better.
>
>
> On 10/15/2016 01:47 AM, Leo Bicknell wrote:
>
>> In a message written on Thu, Oct 13, 2016 at 05:48:18PM +, rar wrote:
>>
>>> The goal is to keep the single BGP router from being a single point of
>>> failure.
>>>
>> I don't really understand the failure analysis / uptime calculation.
>>
>> There is one router on the Comcast side, which is a single point of
>> failure.
>>
>> There is one circuit to your prem, which is a single point of failure.
>>
>> To connect two routers on your end you must terminate the circuit
>> in a switch, which is a single point of failure.
>>
>> And yet, in the face of all that somehow running two routers with
>> two BGP sessions on your end increases your uptime?
>>
>> The only way that would even remotely make sense is if the routers
>> in question were horribly broken / mismanaged so (had to be?) reboot(ed)
>> on a regular basis.  However if uptime is so important using gear
>> with that property makes no sense!
>>
>> I'm pretty sure without actually doing the math that you'll be more
>> reliable with a single quality router (elminiation of complexity),
>> and that if you really need maximum uptime that you had better get
>> a second circuit, on a diverse path, into a different router probably
>> from a different carrier.
>>
>>
>


-- 
Bill Blackford

Logged into reality and abusing my sudo privileges.

Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Paul S.]

+1, could not have said it better.

On 10/15/2016 01:47 AM, Leo Bicknell wrote:

In a message written on Thu, Oct 13, 2016 at 05:48:18PM +, rar wrote:

The goal is to keep the single BGP router from being a single point of failure.

I don't really understand the failure analysis / uptime calculation.

There is one router on the Comcast side, which is a single point of
failure.

There is one circuit to your prem, which is a single point of failure.

To connect two routers on your end you must terminate the circuit
in a switch, which is a single point of failure.

And yet, in the face of all that somehow running two routers with
two BGP sessions on your end increases your uptime?

The only way that would even remotely make sense is if the routers
in question were horribly broken / mismanaged so (had to be?) reboot(ed)
on a regular basis.  However if uptime is so important using gear
with that property makes no sense!

I'm pretty sure without actually doing the math that you'll be more
reliable with a single quality router (elminiation of complexity),
and that if you really need maximum uptime that you had better get
a second circuit, on a diverse path, into a different router probably
from a different carrier.

Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Leo Bicknell]

In a message written on Thu, Oct 13, 2016 at 05:48:18PM +, rar wrote:
> The goal is to keep the single BGP router from being a single point of 
> failure.

I don't really understand the failure analysis / uptime calculation.

There is one router on the Comcast side, which is a single point of
failure.

There is one circuit to your prem, which is a single point of failure.

To connect two routers on your end you must terminate the circuit
in a switch, which is a single point of failure.

And yet, in the face of all that somehow running two routers with
two BGP sessions on your end increases your uptime?

The only way that would even remotely make sense is if the routers
in question were horribly broken / mismanaged so (had to be?) reboot(ed)
on a regular basis.  However if uptime is so important using gear
with that property makes no sense!

I'm pretty sure without actually doing the math that you'll be more
reliable with a single quality router (elminiation of complexity),
and that if you really need maximum uptime that you had better get
a second circuit, on a diverse path, into a different router probably
from a different carrier.

-- 
Leo Bicknell - bickn...@ufp.org
PGP keys at http://www.ufp.org/~bicknell/


pgpgyRLHs65Kl.pgp
Description: PGP signature

Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Ryan, Spencer]

Run your IPv4 peer to one router and IPv6 to another. Boom, redundancy!


Spencer Ryan | Senior Systems Administrator | 
sr...@arbor.net<mailto:sr...@arbor.net>
Arbor Networks
+1.734.794.5033 (d) | +1.734.846.2053 (m)
www.arbornetworks.com<http://www.arbornetworks.com/>



From: NANOG <nanog-boun...@nanog.org> on behalf of Jörg Kost <j...@ip-clear.de>
Sent: Thursday, October 13, 2016 3:59:29 PM
To: rar
Cc: nanog@nanog.org
Subject: Re: Two BGP peering sessions on single Comcast Fiber Connection?


On 13 Oct 2016, at 19:48, rar wrote:

> Comcast said they could not support two separate BGP peering sessions
> on the same circuit.  Does anyone have any counter examples?  We used
> to have this setup with Comcast 5+ years ago, but now they say they
> can't support it.
>

So how do they connect ip6 sessions? ;-)

Jörg

Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Jörg Kost]

On 13 Oct 2016, at 19:48, rar wrote:

Comcast said they could not support two separate BGP peering sessions 
on the same circuit.  Does anyone have any counter examples?  We used 
to have this setup with Comcast 5+ years ago, but now they say they 
can't support it.




So how do they connect ip6 sessions? ;-)

Jörg

Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Dovid Bender]

Whenever we set up a bgp peer we do that to minimize downtime when doing
maint. It's hit or miss. HE required a second physicall connection NTT was
more than accommodating.

On Oct 13, 2016 15:06, "Mike Poublon"  wrote:

> I started a thread around the same topic back on 10/16 of 2014. A Comcast
> engineer (who ultimately spoke to the national product manager) came back
> after discussing and said the same thing "We don't support that". I got a
> slightly longer explanation of:
>
> 
>
> In a nutshell, when we design a product we do it to accommodate the most
> typical customer cases.
> Given that the design includes a single fiber path and thus the fiber path
> and device that terminates on either end each are a single point of
> failure, adding extra BGP sessions doesn’t seem to add value in the typical
> failure scenarios.  In order to achieve the simplest and most scalable
> solution to address the market, we rely on narrowing the possible
> combinations of parameters.
>
> 
>
> I explained to them that their interpretation prevents me from being able
> to do concurrent maintenance on my side (single router reboot/upgrade,
> etc). Never got anywhere with it though.
>
> I'm still interested in having this set up, but have given up on it ever
> really coming to reality. Luckily ALL of my other providers were more than
> happy to set up an extra session.
>
> If anyone from Comcast is listening, there is customer demand for this.
> It's not about making it better for Comcast, it's about allowing customers
> to have more flexibility.
>
> Mike Poublon
>
> /Senior Datacenter Network Engineer/
>
> *Secant Technologies*
>
> 6395 Technology Ave. Suite A
>
> Kalamazoo, MI 49009
>
> On 10/13/2016 1:48 PM, rar wrote:
>
>> After a many month wait, we were ready to turn up our BGP peering
>> sessions on a new Comcast fiber connection.
>>
>> With our other providers (Level 3 and Verizon) we have edge routers that
>> directly connect between the provider's on premise connection and our
>> primary and a backup core routers.  Each core router has a multihop BGP
>> session with the provider's BGP router.  The goal is to keep the single BGP
>> router from being a single point of failure.
>>
>> Comcast said they could not support two separate BGP peering sessions on
>> the same circuit.  Does anyone have any counter examples?  We used to have
>> this setup with Comcast 5+ years ago, but now they say they can't support
>> it.
>>
>>
>> Bob Roswell
>> brosw...@syssrc.com
>> 410-771-5544 ext 4336
>>
>> Computer Museum Highlights
>>
>>
>

Re: Two BGP peering sessions on single Comcast Fiber Connection?

[Mike Poublon]

I started a thread around the same topic back on 10/16 of 2014. A 
Comcast engineer (who ultimately spoke to the national product manager) 
came back after discussing and said the same thing "We don't support 
that". I got a slightly longer explanation of:




In a nutshell, when we design a product we do it to accommodate the most 
typical customer cases.
Given that the design includes a single fiber path and thus the fiber 
path and device that terminates on either end each are a single point of 
failure, adding extra BGP sessions doesn’t seem to add value in the 
typical failure scenarios.  In order to achieve the simplest and most 
scalable solution to address the market, we rely on narrowing the 
possible combinations of parameters.




I explained to them that their interpretation prevents me from being 
able to do concurrent maintenance on my side (single router 
reboot/upgrade, etc). Never got anywhere with it though.


I'm still interested in having this set up, but have given up on it ever 
really coming to reality. Luckily ALL of my other providers were more 
than happy to set up an extra session.


If anyone from Comcast is listening, there is customer demand for this. 
It's not about making it better for Comcast, it's about allowing 
customers to have more flexibility.


Mike Poublon

/Senior Datacenter Network Engineer/

*Secant Technologies*

6395 Technology Ave. Suite A

Kalamazoo, MI 49009

On 10/13/2016 1:48 PM, rar wrote:

After a many month wait, we were ready to turn up our BGP peering sessions on a 
new Comcast fiber connection.

With our other providers (Level 3 and Verizon) we have edge routers that 
directly connect between the provider's on premise connection and our primary 
and a backup core routers.  Each core router has a multihop BGP session with 
the provider's BGP router.  The goal is to keep the single BGP router from 
being a single point of failure.

Comcast said they could not support two separate BGP peering sessions on the 
same circuit.  Does anyone have any counter examples?  We used to have this 
setup with Comcast 5+ years ago, but now they say they can't support it.


Bob Roswell
brosw...@syssrc.com
410-771-5544 ext 4336

Computer Museum Highlights

Two BGP peering sessions on single Comcast Fiber Connection?

[rar]

After a many month wait, we were ready to turn up our BGP peering sessions on a 
new Comcast fiber connection.

With our other providers (Level 3 and Verizon) we have edge routers that 
directly connect between the provider's on premise connection and our primary 
and a backup core routers.  Each core router has a multihop BGP session with 
the provider's BGP router.  The goal is to keep the single BGP router from 
being a single point of failure.

Comcast said they could not support two separate BGP peering sessions on the 
same circuit.  Does anyone have any counter examples?  We used to have this 
setup with Comcast 5+ years ago, but now they say they can't support it.


Bob Roswell
brosw...@syssrc.com
410-771-5544 ext 4336

Computer Museum Highlights