Re: Muni Fiber and Politics

[mcfbbqroast .]

This would be my humble suggestion:

- lines provider runs fibre pair from each home to co. By default the lines
provider installs a simple consumer terminal, with gigabit Ethernet outputs
and POTS.

- lines provider provides a reasonably oversubscribed service to soft hand
over to ISPs (think 96 Gbps lines to 2 10gbps ports). Perhaps upgrading so
such a ratio never becomes congested could be a requirement?

-  lines provider also rents individual lines to ISPs which they can use
directly. Rent should be lower than soft handover.

This way ISPs can easily offer services. POTS over VoIP can be setup on
installation of the terminal (so handover to the ISP is seamless). Finally
business and residential services can also be provided over the fibre
directly (this will be attractive to ISPs with many ports, to reduce costs,
and premium/business ISPs to add control).

- ideally the lines provider would aid in providing cheap backhaul from the
co (while still allowing 3rd party users to bring fibre in).

Sorry for the engrish, I'm on a mobile device :(.
On 1 Aug 2014 17:43, "Mark Tinka"  wrote:

> On Thursday, July 31, 2014 02:01:28 PM Måns Nilsson wrote:
>
> > It is better, both for the customer and the provider.
>
> If the provider is able to deliver 1Gbps to every home
> (either on copper or fibre) with little to no uplink
> oversubscription (think 44x customer-facing Gig-E ports + 4x
> 10Gbps uplink ports), essentially, there is no limit to what
> services a provider and its partners can offer to its
> customers.
>
> Mark.
>

Re: Netflix To Cogent To World

[Mark Tinka]

On Thursday, July 31, 2014 07:10:49 PM Owen DeLong wrote:

> You are still misinterpreting my statement, or at least
> it appears that you are.

The pleasures of e-mail, and tones they do not convey :-)...

> I am not saying that Netflix is attempting to “grab”.

Yes, that's what I meant - by "grab them" I meant the 
traditional ISP's who are also in the content game (U-Verse, 
Xfinity, FiOS, e.t.c.) are making a deliberate or 
unconscious play to "grab" Netflix (where grab means either 
frustrate them or take them out of the equation).

So yes, we are on the same page, Owen; the lack of body in 
e-mail is not either of us justice :-). 

But, like I said, all this is conjecture on my part. Netflix 
may not have the might or time to become an ISP, but other 
networks that play in content (and are currently not known 
for being ISP's, even though they may run very large global 
IP networks) might.

> (Netflix is also a distributor, not owner).

I'm aware - it's easier for me to say "content owner" to 
encompass that industry (given some content players own and 
distribute content, while others simply distribute it), but 
I know this list is well clued in on the nuances of the 
terminology.

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Muni Fiber and Politics

[Mark Tinka]

On Thursday, July 31, 2014 02:01:28 PM Måns Nilsson wrote:

> It is better, both for the customer and the provider.

If the provider is able to deliver 1Gbps to every home 
(either on copper or fibre) with little to no uplink 
oversubscription (think 44x customer-facing Gig-E ports + 4x 
10Gbps uplink ports), essentially, there is no limit to what 
services a provider and its partners can offer to its 
customers.

Mark.


signature.asc
Description: This is a digitally signed message part.

Re: Greenfield Access Network

[Colton Conor]

Scott,

Thank you for your input.

What do you recommend for network segmentation? A VLAN per Chassis, a VLAN
per service, or a VLAN per customer/port? When you say qinq VLANs are you
referring to the CVLAN model?

I am really interested to know how the largest providers, like Comcast,
AT&T, and Verizon setup their residential networks. Little information is
available on the internet besides the access platforms they use.


On Thu, Jul 31, 2014 at 1:51 PM, Scott Helms  wrote:

>
>
> On Thu, Jul 31, 2014 at 2:25 PM, Colton Conor 
> wrote:
>
>> I have read both the Juniper MX and Cisco ASR9K do support this advanced
>> BRAS functionality, what Juniper calls Subscriber Feature Management and
>> what Cisco calls BGN. These software functions run on the router itself,
>> however the are not free or included with the base chassis. To enable these
>> you must pay a hefty fee. So you are saying that these advanced feature
>> packs that the largest networking markers in the world sell are really not
>> needed anymore due to advancements on the access vendor side of the house?
>> From the reading I have done about these solutions, it is kind of like
>> PPPoE with a radius setup, but instead DHCP option 82 with a radius setup.
>> These routers are also capable of running a local DHCP server, but I am not
>> sure if that is recommended.
>>
>
> Yeah, that's it in a nutshell.  There are several options, like matching
> on Option 82 or redirecting to a web page, but at the end of the day I
> don't believe they're worth the time or expense.  Keep in mind that earlier
> in my career I was a huge proponent of BRAS architecture and I've put in
> everything from Nortel Shasta's to Lucent Terminators, to Redbacks, to
> Juniper ERXs and several more models I can't remember.  Once you get past
> the whole lack of authentication, which was never very secure, and
> understand that you can depend on Option 82 to tell you where a session
> came from physically the rest is just finding away to count and account for
> bits.
>
> Oh, and I never recommend running the DHCP daemon on a piece of networking
> gear for service providers.
>
>
>>
>> The DPoE DOCSIS provisioning of your GPON network is interesting, but is
>> that really relevant for a new provider if they don't have cable CMTS
>> systems already deployed. Sure, it makes sense for the cable compaines who
>> have already bought billing systems and are used to living in
>> a DOCSIS world. But if you were starting fresh from the group up are you
>> recommending we look at GPON providers like Calix because they support DPoE
>> so we can buy DOCIS billing systems? That is an interesting concept.
>>
>
> I'd strongly recommend finding a vendor that says they will support it on
> the shelves you're going to buy even if they don't today.  Even if you're
> not doing DOCSIS cable modems and don't ever plan to the provisioning
> paradigm (DHCP, TFTP, ToD) is much simpler than the proprietary north bound
> (usually SOAP) API that direct integration requires.  You can even build
> your own provisioning system with a little scripting and there are many
> more commercial options than there are for direct integration to the
> shelves.
>
>
>>
>>
>>
>>
>>
>> On Thu, Jul 31, 2014 at 12:59 PM, Scott Helms  wrote:
>>
>>>
>>> On Thu, Jul 31, 2014 at 12:07 PM, Colton Conor 
>>> wrote:
>>>
 Scott,

 Thanks for the long post.

 We will use a layer 2 10G aggregation switch then to aggregate the
 chassis at the core location. Do you have any recommendations on 10G
 switches?

>>>
>>> Not really, just stick with one of the major brands and you _should_ be
>>> fine.
>>>
>>>

 Yes I realize the math is a little backwards as this is
 all hypothetical at this point. We would provision each ONT as a shared
 1Gbps offering similar to Google Fiber. We know there will be a large
 amount of oversubscription as no one really uses a full Gbps or anywhere
 close to it. I just wanted to stress the point that carrier redundancy at
 the 10G level would be a requirement for the core router, and it should of
 course have 10G links going to the uplinks on the aggregation switch. I
 think the Cisco ASR9k and the Juniper MX line will do well. Not sure if
 there are any others that can handle this level of traffic on the BGP side?

>>>
>>> That's reasonable IMO and yes, I think the Juniper MX can handle that as
>>> well as some other functions for you related to subscriber management if
>>> you want.  The MX line has a full BRAS set of capabilities built into it
>>> that it inherited from the older ERX line, but they're commonly deployed
>>> without using any of them of as well.
>>>
>>>

 So we have a 10G aggregation switch to aggregate the chassis uplink
 connections, and a 10G router BGP capable router.

 I really liked your article on DHCP vs PPP for DSL networks. We
 definitely agree the way to go is with a DHCP server. A cou

Re: Greenfield Access Network

[Scott Helms]

On Thu, Jul 31, 2014 at 2:25 PM, Colton Conor 
wrote:

> I have read both the Juniper MX and Cisco ASR9K do support this advanced
> BRAS functionality, what Juniper calls Subscriber Feature Management and
> what Cisco calls BGN. These software functions run on the router itself,
> however the are not free or included with the base chassis. To enable these
> you must pay a hefty fee. So you are saying that these advanced feature
> packs that the largest networking markers in the world sell are really not
> needed anymore due to advancements on the access vendor side of the house?
> From the reading I have done about these solutions, it is kind of like
> PPPoE with a radius setup, but instead DHCP option 82 with a radius setup.
> These routers are also capable of running a local DHCP server, but I am not
> sure if that is recommended.
>

Yeah, that's it in a nutshell.  There are several options, like matching on
Option 82 or redirecting to a web page, but at the end of the day I don't
believe they're worth the time or expense.  Keep in mind that earlier in my
career I was a huge proponent of BRAS architecture and I've put in
everything from Nortel Shasta's to Lucent Terminators, to Redbacks, to
Juniper ERXs and several more models I can't remember.  Once you get past
the whole lack of authentication, which was never very secure, and
understand that you can depend on Option 82 to tell you where a session
came from physically the rest is just finding away to count and account for
bits.

Oh, and I never recommend running the DHCP daemon on a piece of networking
gear for service providers.


>
> The DPoE DOCSIS provisioning of your GPON network is interesting, but is
> that really relevant for a new provider if they don't have cable CMTS
> systems already deployed. Sure, it makes sense for the cable compaines who
> have already bought billing systems and are used to living in
> a DOCSIS world. But if you were starting fresh from the group up are you
> recommending we look at GPON providers like Calix because they support DPoE
> so we can buy DOCIS billing systems? That is an interesting concept.
>

I'd strongly recommend finding a vendor that says they will support it on
the shelves you're going to buy even if they don't today.  Even if you're
not doing DOCSIS cable modems and don't ever plan to the provisioning
paradigm (DHCP, TFTP, ToD) is much simpler than the proprietary north bound
(usually SOAP) API that direct integration requires.  You can even build
your own provisioning system with a little scripting and there are many
more commercial options than there are for direct integration to the
shelves.


>
>
>
>
>
> On Thu, Jul 31, 2014 at 12:59 PM, Scott Helms  wrote:
>
>>
>> On Thu, Jul 31, 2014 at 12:07 PM, Colton Conor 
>> wrote:
>>
>>> Scott,
>>>
>>> Thanks for the long post.
>>>
>>> We will use a layer 2 10G aggregation switch then to aggregate the
>>> chassis at the core location. Do you have any recommendations on 10G
>>> switches?
>>>
>>
>> Not really, just stick with one of the major brands and you _should_ be
>> fine.
>>
>>
>>>
>>> Yes I realize the math is a little backwards as this is all hypothetical
>>> at this point. We would provision each ONT as a shared 1Gbps offering
>>> similar to Google Fiber. We know there will be a large amount of
>>> oversubscription as no one really uses a full Gbps or anywhere close to it.
>>> I just wanted to stress the point that carrier redundancy at the 10G level
>>> would be a requirement for the core router, and it should of course have
>>> 10G links going to the uplinks on the aggregation switch. I think the Cisco
>>> ASR9k and the Juniper MX line will do well. Not sure if there are any
>>> others that can handle this level of traffic on the BGP side?
>>>
>>
>> That's reasonable IMO and yes, I think the Juniper MX can handle that as
>> well as some other functions for you related to subscriber management if
>> you want.  The MX line has a full BRAS set of capabilities built into it
>> that it inherited from the older ERX line, but they're commonly deployed
>> without using any of them of as well.
>>
>>
>>>
>>> So we have a 10G aggregation switch to aggregate the chassis uplink
>>> connections, and a 10G router BGP capable router.
>>>
>>> I really liked your article on DHCP vs PPP for DSL networks. We
>>> definitely agree the way to go is with a DHCP server. A couple of items
>>> your article left as big questions:
>>>
>>
>>
>>> 1. The article mentioned DHCP doesn't do the other part of what PPPoE or
>>> PPPoA does, which is generate RADIUS accounting records that give us the
>>> bandwidth information. So that’s one of the main challenges in switching to
>>> a DHCP based system. So, how do you handle bandwidth tracking in an all
>>> DHCP environment then? If I want to track how many GB a customer used last
>>> month, or the average Mbps used how do you do so?
>>>
>>
>> There are a few ways to get at that problem.  You can use Netflow/IPFIX
>> collection to gat

Re: Greenfield Access Network

[Colton Conor]

I have read both the Juniper MX and Cisco ASR9K do support this advanced
BRAS functionality, what Juniper calls Subscriber Feature Management and
what Cisco calls BGN. These software functions run on the router itself,
however the are not free or included with the base chassis. To enable these
you must pay a hefty fee. So you are saying that these advanced feature
packs that the largest networking markers in the world sell are really not
needed anymore due to advancements on the access vendor side of the house?
>From the reading I have done about these solutions, it is kind of like
PPPoE with a radius setup, but instead DHCP option 82 with a radius setup.
These routers are also capable of running a local DHCP server, but I am not
sure if that is recommended.

The DPoE DOCSIS provisioning of your GPON network is interesting, but is
that really relevant for a new provider if they don't have cable CMTS
systems already deployed. Sure, it makes sense for the cable compaines who
have already bought billing systems and are used to living in
a DOCSIS world. But if you were starting fresh from the group up are you
recommending we look at GPON providers like Calix because they support DPoE
so we can buy DOCIS billing systems? That is an interesting concept.




On Thu, Jul 31, 2014 at 12:59 PM, Scott Helms  wrote:

>
> On Thu, Jul 31, 2014 at 12:07 PM, Colton Conor 
> wrote:
>
>> Scott,
>>
>> Thanks for the long post.
>>
>> We will use a layer 2 10G aggregation switch then to aggregate the
>> chassis at the core location. Do you have any recommendations on 10G
>> switches?
>>
>
> Not really, just stick with one of the major brands and you _should_ be
> fine.
>
>
>>
>> Yes I realize the math is a little backwards as this is all hypothetical
>> at this point. We would provision each ONT as a shared 1Gbps offering
>> similar to Google Fiber. We know there will be a large amount of
>> oversubscription as no one really uses a full Gbps or anywhere close to it.
>> I just wanted to stress the point that carrier redundancy at the 10G level
>> would be a requirement for the core router, and it should of course have
>> 10G links going to the uplinks on the aggregation switch. I think the Cisco
>> ASR9k and the Juniper MX line will do well. Not sure if there are any
>> others that can handle this level of traffic on the BGP side?
>>
>
> That's reasonable IMO and yes, I think the Juniper MX can handle that as
> well as some other functions for you related to subscriber management if
> you want.  The MX line has a full BRAS set of capabilities built into it
> that it inherited from the older ERX line, but they're commonly deployed
> without using any of them of as well.
>
>
>>
>> So we have a 10G aggregation switch to aggregate the chassis uplink
>> connections, and a 10G router BGP capable router.
>>
>> I really liked your article on DHCP vs PPP for DSL networks. We
>> definitely agree the way to go is with a DHCP server. A couple of items
>> your article left as big questions:
>>
>
>
>> 1. The article mentioned DHCP doesn't do the other part of what PPPoE or
>> PPPoA does, which is generate RADIUS accounting records that give us the
>> bandwidth information. So that’s one of the main challenges in switching to
>> a DHCP based system. So, how do you handle bandwidth tracking in an all
>> DHCP environment then? If I want to track how many GB a customer used last
>> month, or the average Mbps used how do you do so?
>>
>
> There are a few ways to get at that problem.  You can use Netflow/IPFIX
> collection to gather the usage from your router, accepting that you're only
> going to get information on layer 3 traffic, which generally isn't a
> problem.  You will need to match the IPs up against your Option 82 parsing
> which will give you the circuit ID, IP address, and WAN MAC of the ONT.
>  You can also poll your shelves via SNMP, CLI, TL-1, and/or Netconf to
> collect the data and put it into a database in much the same way you can
> use RADIUS accounting data.
>
>
>>
>> 2. I liked your option 82 example, and that works well for DSL networks
>> where one port is tied to one customer. But how does option 82 work when
>> you have multiple customers hanging off a GPON port? What does GPON use a
>> subport identifier?
>>
>
> Yep, the different vendors implement it slightly differently, usually the
> ONT MAC/serial will be included or the ONT ID will be included.  Talk with
> your vendor, all the major OLT vendors are very familiar with Option 82 and
> in many cases they can tailor what their boxes send to make it easier for
> you.
>
>
>
>> 3. You mentioned, DHCP is again, not a authentication protocol. So what
>> handles authentication then if only DHCP is used, and there are no
>> usernames and passwords? I guess for DSL networks you can enable or disable
>> the port to allow or disallow access, and Option 82 for identification? I
>> assume you wouldn't want to shut off the GPON OLT port if one customer
>> wasn't paying their bill a

Re: Greenfield Access Network

[Scott Helms]

On Thu, Jul 31, 2014 at 12:07 PM, Colton Conor 
wrote:

> Scott,
>
> Thanks for the long post.
>
> We will use a layer 2 10G aggregation switch then to aggregate the chassis
> at the core location. Do you have any recommendations on 10G switches?
>

Not really, just stick with one of the major brands and you _should_ be
fine.


>
> Yes I realize the math is a little backwards as this is all hypothetical
> at this point. We would provision each ONT as a shared 1Gbps offering
> similar to Google Fiber. We know there will be a large amount of
> oversubscription as no one really uses a full Gbps or anywhere close to it.
> I just wanted to stress the point that carrier redundancy at the 10G level
> would be a requirement for the core router, and it should of course have
> 10G links going to the uplinks on the aggregation switch. I think the Cisco
> ASR9k and the Juniper MX line will do well. Not sure if there are any
> others that can handle this level of traffic on the BGP side?
>

That's reasonable IMO and yes, I think the Juniper MX can handle that as
well as some other functions for you related to subscriber management if
you want.  The MX line has a full BRAS set of capabilities built into it
that it inherited from the older ERX line, but they're commonly deployed
without using any of them of as well.


>
> So we have a 10G aggregation switch to aggregate the chassis uplink
> connections, and a 10G router BGP capable router.
>
> I really liked your article on DHCP vs PPP for DSL networks. We definitely
> agree the way to go is with a DHCP server. A couple of items your article
> left as big questions:
>


> 1. The article mentioned DHCP doesn't do the other part of what PPPoE or
> PPPoA does, which is generate RADIUS accounting records that give us the
> bandwidth information. So that’s one of the main challenges in switching to
> a DHCP based system. So, how do you handle bandwidth tracking in an all
> DHCP environment then? If I want to track how many GB a customer used last
> month, or the average Mbps used how do you do so?
>

There are a few ways to get at that problem.  You can use Netflow/IPFIX
collection to gather the usage from your router, accepting that you're only
going to get information on layer 3 traffic, which generally isn't a
problem.  You will need to match the IPs up against your Option 82 parsing
which will give you the circuit ID, IP address, and WAN MAC of the ONT.
 You can also poll your shelves via SNMP, CLI, TL-1, and/or Netconf to
collect the data and put it into a database in much the same way you can
use RADIUS accounting data.


>
> 2. I liked your option 82 example, and that works well for DSL networks
> where one port is tied to one customer. But how does option 82 work when
> you have multiple customers hanging off a GPON port? What does GPON use a
> subport identifier?
>

Yep, the different vendors implement it slightly differently, usually the
ONT MAC/serial will be included or the ONT ID will be included.  Talk with
your vendor, all the major OLT vendors are very familiar with Option 82 and
in many cases they can tailor what their boxes send to make it easier for
you.



> 3. You mentioned, DHCP is again, not a authentication protocol. So what
> handles authentication then if only DHCP is used, and there are no
> usernames and passwords? I guess for DSL networks you can enable or disable
> the port to allow or disallow access, and Option 82 for identification? I
> assume you wouldn't want to shut off the GPON OLT port if one customer
> wasn't paying their bill as it would affect the other customers on that
> port. I assume access vendors allow you to shut down the sub port or ONT in
> this situation for GPON? Still that seems messy having to login to a shelf
> or EMS system or API to an EMS system especially if you have multiple
> access vendors in a network. Is there a way to do authentication with DHCP?
> What about open networks like wifi where anyone can connect, so you don't
> have the ability to turn of the port or disable the end device?
> 4. I don't think anyone is buying a BRAS anymore, but looks like Cisco,
> Juniper, and ALU have what they call BGN, Broadband Subscriber Management,
> and other similar software. How are these different from BRAS functionality?
>

First, if you can manage it turn on DOCSIS provisioning of your GPON
network.  AFAIK only Calix has announced this functionality, but I expect
the others to follow suit now that there is an official effort at CableLabs
to allow that.

http://www.lightreading.com/cable-video/docsis/calix-launches-docsis-provisioning-of-gpon/d/d-id/709859

The notion of managing ports and profiles via (an ever changing) shelf API
is one of the main reasons that telco billing systems cost so much compared
to cable billing systems.  If you can't swing DPoG then you're kind of
stuck, either you can implement the API your vendor supplies with your
billing system, manage the profile assignment manually (yuck), or just
provisi

Re: EBAY reachability issues

[Mike A]

On Fri, Aug 01, 2014 at 02:23:54AM +0900, Paul S. wrote:
> Appears to be loading just fine from here in Sg.
> On Jul 31, 2014 11:21 PM, "Mike A"  wrote:
> 
> > On Thu, Jul 31, 2014 at 02:38:13PM +, Drew Weaver wrote:
> > > We've been seeing some issues with getting to Ebay this morning, only a
> > very select few of their GSLB sites in DNS seem to be responding (to us at
> > least)...
> > >
> > > Connecting to www.ebay.com|66.135.210.181|:80... connected.
> > > HTTP request sent, awaiting response... 200 OK
> > >
> > > Connecting to www.ebay.com|66.211.181.161|:80... connected.
> > > HTTP request sent, awaiting response...timed out
> > >
> > > Connecting to www.ebay.com|66.211.181.181|:80... connected.
> > > HTTP request sent, awaiting response...timed out
> > >
> > > Tried reaching out to them but can't contact anyone if anyone has a
> > contact there please forward.
> >
> > isitdownrightnow.com says that ebay isn't answering and hasn't been for
> > close to 14 hours.

The outage appears to be provider-specific, according to comments in that
particular thread at isitdownrightnow. And eBay appears to be flapping; it was
up for a short time, and now has been down for 21+ minutes. Glad it's not my
site.

-- 
Mike Andrews, W5EGO
mi...@mikea.ath.cx
Tired old sysadmin 

Re: EBAY reachability issues

[Paul S.]

Appears to be loading just fine from here in Sg.
On Jul 31, 2014 11:21 PM, "Mike A"  wrote:

> On Thu, Jul 31, 2014 at 02:38:13PM +, Drew Weaver wrote:
> > We've been seeing some issues with getting to Ebay this morning, only a
> very select few of their GSLB sites in DNS seem to be responding (to us at
> least)...
> >
> > Connecting to www.ebay.com|66.135.210.181|:80... connected.
> > HTTP request sent, awaiting response... 200 OK
> >
> > Connecting to www.ebay.com|66.211.181.161|:80... connected.
> > HTTP request sent, awaiting response...timed out
> >
> > Connecting to www.ebay.com|66.211.181.181|:80... connected.
> > HTTP request sent, awaiting response...timed out
> >
> > Tried reaching out to them but can't contact anyone if anyone has a
> contact there please forward.
>
> isitdownrightnow.com says that ebay isn't answering and hasn't been for
> close to 14 hours.
>
> --
> Mike Andrews, W5EGO
> mi...@mikea.ath.cx
> Tired old sysadmin
>

Re: Greenfield Access Network

[William Herrin]

On Thu, Jul 31, 2014 at 9:23 AM, Colton Conor  wrote:
> If a new operator or city is building a greenfield access network from the
> ground up,

Hi Colton,

We just had a long discussion in this forum to the effect that if a
city builds a greenfield access network, it would be best limited to
"layer 1" services. That is, deliver dark fiber and invite as many
service providers as possible to light it with whatever services
they're inclined to sell. Commercially, the L1 infrastructure presents
the barrier to entry. That's why you don't have enough competitive
commercial entities mooting the need to even discuss providing
Internet as a municipal service. Even the smallest city is attractive
to competitive commercial service providers when they can lease
in-place L1 infrastructure ad hoc.

This isn't as sexy as delivering gigabit Internet in the way roads
aren't as sexy as the cars which drive on them but it relieves the
city of having to make most of the hard-to-get-right decisions that
could tank your effort and turn it into a boondoggle. Let commercial
entities worry about what car will be popular next year and let
commercial entities figure out which stores folks will drive those
cars to. Just worry about where to build roads.




On Thu, Jul 31, 2014 at 12:24 PM, Roland Dobbins  wrote:
> I'm not criticizing you; I'm just trying to make the point that instead
> of concentrating on vendors and technologies and hardware and
> software, it's better to concentrate on *people* who have the
> requisite experience and expertise, and go from there.

This. So much this.

Regards,
Bill Herrin

-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 
Can I solve your unusual networking challenges?

Re: EBAY reachability issues

[Mike A]

On Thu, Jul 31, 2014 at 02:38:13PM +, Drew Weaver wrote:
> We've been seeing some issues with getting to Ebay this morning, only a very 
> select few of their GSLB sites in DNS seem to be responding (to us at 
> least)...
> 
> Connecting to www.ebay.com|66.135.210.181|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> 
> Connecting to www.ebay.com|66.211.181.161|:80... connected.
> HTTP request sent, awaiting response...timed out
> 
> Connecting to www.ebay.com|66.211.181.181|:80... connected.
> HTTP request sent, awaiting response...timed out
> 
> Tried reaching out to them but can't contact anyone if anyone has a contact 
> there please forward.

isitdownrightnow.com says that ebay isn't answering and hasn't been for
close to 14 hours. 

-- 
Mike Andrews, W5EGO
mi...@mikea.ath.cx
Tired old sysadmin 

Re: Netflix To Cogent To World

[Owen DeLong]

On Jul 30, 2014, at 11:21 PM, Mark Tinka  wrote:

> On Thursday, July 31, 2014 01:35:32 AM Owen DeLong wrote:
> 
>> In that case, I would argue that the attempts to freeze
>> Netflix out in a SlowLane extortion scheme are a move by
>> the existing content/ISP conglomerates to do just
>> exactly that, no?
> 
> For Netflix, I can't say for sure whether it is a deliberate 
> attempt to "grab" them or not, but yes, the current 
> environment certainly has the makings of what it would take 
> to achieve such an outcome, on purpose or by accident.
> 
> That said, as successful as Netflix are, I believe there are 
> other content owners out there that are more likely to 
> challenge the traditional ISP model that Netflix ever could. 
> Those are the ones I'm more concerned about.
> 
> Mark.

You are still misinterpreting my statement, or at least it appears
that you are.

I am not saying that Netflix is attempting to “grab”. I’m saying that
Netflix is the target as the content distributors that are already ISPs
attempt to prevent Netflix from capturing more of their content
customers. (Netflix is also a distributor, not owner).

AT&T, Verizon, et. al. also have content businesses and it appears
to me that this battle is an attempt by them to reduce Netflix’
inroads into that business.

IOW, the incumbent ISPs seem to be doing exactly what you described
as an effort to protect their traditional content businesses.

Owen

Re: Greenfield Access Network

[Colton Conor]

Roland,

I agree with everything you mentioned in your email. No matter how much
money and resources you have, if you don't have the talent and people
required to get the job done the project will fail. There a many outfits,
like Scotts for example, that will handle most all of these issues for an
operator that doesn't have the skills, talent, or personnel to deploy such
a network on their own.

I tried to keep the topics as broad as possible. No, I didn't go into
detail about recursive or authoritative as I figured the general term DNS
would cover both for the readers of this forum. The same with availability
and resiliency and telemetry visibility and network hardening and the other
detailed terms you have mentioned as I am making the assumption that this
networking gear being talk about (carrier grade routers) would have most of
these capabilities and people that would implement them (certified network
engineers) would handle these issues.

With that being said, we are not trying to crowdsource the architecture,
design, deployment, and operations of our network. We are just seeking
categorical
advice as mentioned. If you ask this question to many of the network
vendors that make these products they will try to oversell you on items you
don't need. Just trying to cut through some of the marketing BS that the
vendors produce, and see what people in the real world are actually
deploying.




On Thu, Jul 31, 2014 at 11:24 AM, Roland Dobbins  wrote:

>
> On Jul 31, 2014, at 8:23 PM, Colton Conor  wrote:
>
> > Is a firewall needed in the core?
>
> No, quite the opposite:
>
> 
>
> > How would you build a access network from the ground up if you had the
> resources and time to do so?
>
> I'd hire folks who have experience from both and architectural and
> operational perspectives, and who have the necessary local knowledge.  Most
> of the question you're asking (except the one about iatrogenic stateful
> firewalls) are situationally-specific, and aren't really going to be
> answerable in detail via a mailing-list, no matter the depth and breadth of
> expertise of many of those participating in said email list.
>
> For example, you've asked nothing specifically about recursive or
> authoritative DNS infrastructure, although they're both key (you did
> mention DNS generically, which is good, but that's overly broad).  Nothing
> about availability and resiliency and telemetry visibility and network
> hardening.  Nothing about access policies, mitigation systems, quarantine
> systems, etc.  Nothing about upstream transit requirements, nothing about
> peering goals and imperatives.  Nothing about redundancy at any level/in
> any area/for any function.  And so forth.
>
> I'm not criticizing you; I'm just trying to make the point that instead of
> concentrating on vendors and technologies and hardware and software, it's
> better to concentrate on *people* who have the requisite experience and
> expertise, and go from there.  There are lots of specializations and
> subspecializations, and it's important to have folks who have broad
> experience spanning multiple areas, as well as others who know *everything*
> in a given area.
>
> While you can get some categorical advice, you can't really crowdsource
> the architecture, design, deployment, and operations of your network.
>
> ;>
>
> --
> Roland Dobbins  // 
>
>Equo ne credite, Teucri.
>
>   -- Laocoön
>
>

Re: Greenfield Access Network

[Roland Dobbins]

On Jul 31, 2014, at 8:23 PM, Colton Conor  wrote:

> Is a firewall needed in the core?

No, quite the opposite:



> How would you build a access network from the ground up if you had the 
> resources and time to do so?

I'd hire folks who have experience from both and architectural and operational 
perspectives, and who have the necessary local knowledge.  Most of the question 
you're asking (except the one about iatrogenic stateful firewalls) are 
situationally-specific, and aren't really going to be answerable in detail via 
a mailing-list, no matter the depth and breadth of expertise of many of those 
participating in said email list.

For example, you've asked nothing specifically about recursive or authoritative 
DNS infrastructure, although they're both key (you did mention DNS generically, 
which is good, but that's overly broad).  Nothing about availability and 
resiliency and telemetry visibility and network hardening.  Nothing about 
access policies, mitigation systems, quarantine systems, etc.  Nothing about 
upstream transit requirements, nothing about peering goals and imperatives.  
Nothing about redundancy at any level/in any area/for any function.  And so 
forth.

I'm not criticizing you; I'm just trying to make the point that instead of 
concentrating on vendors and technologies and hardware and software, it's 
better to concentrate on *people* who have the requisite experience and 
expertise, and go from there.  There are lots of specializations and 
subspecializations, and it's important to have folks who have broad experience 
spanning multiple areas, as well as others who know *everything* in a given 
area.

While you can get some categorical advice, you can't really crowdsource the 
architecture, design, deployment, and operations of your network.

;>

--
Roland Dobbins  // 

   Equo ne credite, Teucri.

  -- Laocoön

Re: Greenfield Access Network

[Colton Conor]

Scott,

Thanks for the long post.

We will use a layer 2 10G aggregation switch then to aggregate the chassis
at the core location. Do you have any recommendations on 10G switches?

Yes I realize the math is a little backwards as this is all hypothetical at
this point. We would provision each ONT as a shared 1Gbps offering similar
to Google Fiber. We know there will be a large amount of oversubscription
as no one really uses a full Gbps or anywhere close to it. I just wanted to
stress the point that carrier redundancy at the 10G level would be a
requirement for the core router, and it should of course have 10G links
going to the uplinks on the aggregation switch. I think the Cisco ASR9k and
the Juniper MX line will do well. Not sure if there are any others that can
handle this level of traffic on the BGP side?

So we have a 10G aggregation switch to aggregate the chassis uplink
connections, and a 10G router BGP capable router.

I really liked your article on DHCP vs PPP for DSL networks. We definitely
agree the way to go is with a DHCP server. A couple of items your article
left as big questions:
1. The article mentioned DHCP doesn't do the other part of what PPPoE or
PPPoA does, which is generate RADIUS accounting records that give us the
bandwidth information. So that’s one of the main challenges in switching to
a DHCP based system. So, how do you handle bandwidth tracking in an all
DHCP environment then? If I want to track how many GB a customer used last
month, or the average Mbps used how do you do so?
2. I liked your option 82 example, and that works well for DSL networks
where one port is tied to one customer. But how does option 82 work when
you have multiple customers hanging off a GPON port? What does GPON use a
subport identifier?
3. You mentioned, DHCP is again, not a authentication protocol. So what
handles authentication then if only DHCP is used, and there are no
usernames and passwords? I guess for DSL networks you can enable or disable
the port to allow or disallow access, and Option 82 for identification? I
assume you wouldn't want to shut off the GPON OLT port if one customer
wasn't paying their bill as it would affect the other customers on that
port. I assume access vendors allow you to shut down the sub port or ONT in
this situation for GPON? Still that seems messy having to login to a shelf
or EMS system or API to an EMS system especially if you have multiple
access vendors in a network. Is there a way to do authentication with DHCP?
What about open networks like wifi where anyone can connect, so you don't
have the ability to turn of the port or disable the end device?
4. I don't think anyone is buying a BRAS anymore, but looks like Cisco,
Juniper, and ALU have what they call BGN, Broadband Subscriber Management,
and other similar software. How are these different from BRAS functionality?

So it looks like there are open source and commercial solutions for DHCP
and DNS. Some providers like Infloblox seems to integrate all these into
one.

So if we have a core router that speaks BGP, a 10G aggregation switch to
aggregate the the chassis, and a device like Infloblox or the other
commercial solutions you mentioned that do DHCP/DNS, is there anything else
that is needed besides the access gear already mentioned in the
assumptions?  Are these large and expensive commercial BGN/Broadband
Subscriber management products a thing of the past or still very relevant
in todays environment?









On Thu, Jul 31, 2014 at 8:54 AM, Scott Helms  wrote:

> "What is the ideal way to aggregate the 40 10G connections from the uplinks
> of the chassis? I would guess a 10G switch since 10G ports on a router
> would be much more expensive?"
>
> Definitely aggregate into a switch first unless you want to run a Layer 3
> switch as your router, which I don't recommend.
>
>
> "Which router is recommended to handle 4 10G internet connections with full
> tables, and then at least 4 10G ports going back to the 10G aggregation
> switch?"
>
> Your math is a little backwards, its very unlikely that you're going to
> have 40 Gbps of Internet (or other interconnection) for the router to
> actually have to process.  What is the average provisioned speed for each
> of the 10k PON ports?  What over subscription rate are you planning for?
>  What, if anything, will you be carrying on net, ie bandwidth consumption
> that won't come from or go to the public Internet?  Your own video, voice,
> or other service are examples of things that are often on net.  In any case
> you're probably in the ASR family with Cisco and I can't remember
> the equivalent from Juniper.
>
>
> How do you handle IP address management? a /20 is only 4096 IP addresses,
> but the network would have potentially 10,000 customers. Assume that
> getting more space from ARIN is not an option. Is CGN an option?
>
> CGN is the option of last resort IMO, but you may have to consider it.  A
> better approach is to see if your backbone providers will agree to 

EBAY reachability issues

[Drew Weaver]

We've been seeing some issues with getting to Ebay this morning, only a very 
select few of their GSLB sites in DNS seem to be responding (to us at least)...

Connecting to www.ebay.com|66.135.210.181|:80... connected.
HTTP request sent, awaiting response... 200 OK

Connecting to www.ebay.com|66.211.181.161|:80... connected.
HTTP request sent, awaiting response...timed out

Connecting to www.ebay.com|66.211.181.181|:80... connected.
HTTP request sent, awaiting response...timed out

Tried reaching out to them but can't contact anyone if anyone has a contact 
there please forward.

Thanks,
-Drew

Re: Greenfield Access Network

[Scott Helms]

"What is the ideal way to aggregate the 40 10G connections from the uplinks
of the chassis? I would guess a 10G switch since 10G ports on a router
would be much more expensive?"

Definitely aggregate into a switch first unless you want to run a Layer 3
switch as your router, which I don't recommend.


"Which router is recommended to handle 4 10G internet connections with full
tables, and then at least 4 10G ports going back to the 10G aggregation
switch?"

Your math is a little backwards, its very unlikely that you're going to
have 40 Gbps of Internet (or other interconnection) for the router to
actually have to process.  What is the average provisioned speed for each
of the 10k PON ports?  What over subscription rate are you planning for?
 What, if anything, will you be carrying on net, ie bandwidth consumption
that won't come from or go to the public Internet?  Your own video, voice,
or other service are examples of things that are often on net.  In any case
you're probably in the ASR family with Cisco and I can't remember
the equivalent from Juniper.


How do you handle IP address management? a /20 is only 4096 IP addresses,
but the network would have potentially 10,000 customers. Assume that
getting more space from ARIN is not an option. Is CGN an option?

CGN is the option of last resort IMO, but you may have to consider it.  A
better approach is to see if your backbone providers will agree to give
some blocks that you can announce and use those blocks for dynamic
customers only.  Your static IP customers should come from your direct ARIN
allotment in case you need to choose a new backbone provider, which is
extremely common over time.


"Dynamic IP
addresses? DHCP?"

DHCP with enforcement from the shelves.  All the major OLT vendors support
doing this so that a customer can only use the address assigned to him by
DHCP and nothing else, except for those customers that you choose to hard
code.  Make most of your "static" customers actually DHCP reservations and
only hard code those that you must.

"How do you separate users and traffic? VLANs, Service VLANs, Per Customer
VLANs, Usernames? Passwords? PPPoE? MAC Separation?
Is a BRAS or BGN functionally really needed or are these older concepts?"

DHCP, with Option 82 logging for the circuit ID is the better path than a
BRAS (PPPoE) these days.  Here's a paper we put together on that topic a
while back:

http://www.zcorum.com/wp-content/uploads/Why-Should-I-Move-from-PPPoA-or-PPPoE-to-DHCP.pdf

Depending on your OLT vendor you can either use their built in port
isolation or QinQ tagging, both are reliable and scalable, just ask your
vendor which is the best option for your specific gear.



"If CGNAT or DHCP is needed, what will host the CGNAT or DHCP service? The
core router, a linux box, or something else?"

I wouldn't have those two services connected personally, though there are
hooks for some of the CGN boxes to talk to DHCP servers.  I would hope you
can get another 6k addresses and avoid the need for CGN altogether.  Having
said that, have you tested your OLTs and ONTs for IPv6 interoperability?
 If they don't handle it well then you're going to have to think about
alternatives like 6RD (http://en.wikipedia.org/wiki/IPv6_rapid_deployment)

For DHCP at your scale you can run ISC DHCP (
http://www.isc.org/downloads/dhcp/) which is the most common open source
DHCP daemon if you someone who can take care of a Linux server, parse the
Option 82 information for logging, and handle the configuration of the DHCP
daemon itself.  Otherwise you might want to look at commercial products
designed for the service provider market like Incongito's BCC and Cisco's
BAC (CNR replacement)

http://www.incognito.com/products/broadband-command-center/
http://www.cisco.com/c/en/us/products/cloud-systems-management/broadband-access-center/index.html


"What about DNS?
Is a firewall needed in the core?
What else is needed?"

There are two kinds of DNS, caching (recursive) and authoritative.  The
first is what your customers will use to resolve things on the Internet and
the second is used to provide caching name servers on the Internet with
information about domains you control (are authoritative for).  The first
needs good performance, availability, and scalability since your customers
will use your caching name servers constantly.  Most people can run BIND at
your scale, again if you have someone with Linux experience, but there are
other alternatives.  PowerDNS has both caching and authoritative modules
and there are some commercial offerings out there both as cloud hosting and
local deployments.  Your backbone provider will also often have caching
name servers your customers can use, but the quality varies quite a bit.
 You can also, especially at first, leverage some of the free offerings
like Google's DNS.  I don't recommend firewalls for service provider
networks, but you should make sure your gear can run (and is configured to
do so) BCP 38.


Scott Helms
Vice Presi

Greenfield Access Network

[Colton Conor]

If a new operator or city is building a greenfield access network from the
ground up, what software and hardware is needed in the core network to
provide and manage residential and business internet services similar to
the likes of AT&T, Comcast, and Google Fiber? Television and Telephone
services are not to be considered only internet.

Assume hypothetically the operator already has the following in place:
10 GPON OLTs Chassis from an access vendor in 10 POPs around town (each POP
has 1 Chassis). Each OLT Chassis has 4 10G Uplinks back to the core.
Dark fiber going from the POP locations back to the core location
Assume a 32:1 way split, and each OLT chassis has enough ports populated to
serve the area.
10,000 GPON ONTs. The ONTs can be put in routed gateway or bridged mode.
Assume you are building a network designed to serve 10,000 subs
All the fiber splitters, ducts, fiber, etc connecting the OLTs to the ONTs
is already in place
ASN from ARIN
/20 of IPv4 space and /32 of IPv6 space from ARIN
4 burstable 10G internet connections from 4 tier 1 internet providers

Questions are:

What is the ideal way to aggregate the 40 10G connections from the uplinks
of the chassis? I would guess a 10G switch since 10G ports on a router
would be much more expensive?
Which router is recommended to handle 4 10G internet connections with full
tables, and then at least 4 10G ports going back to the 10G aggregation
switch?
How do you handle IP address management? a /20 is only 4096 IP addresses,
but the network would have potentially 10,000 customers. Assume that
getting more space from ARIN is not an option. Is CGN an option? Dynamic IP
addresses? DHCP?
How do you separate users and traffic? VLANs, Service VLANs, Per Customer
VLANs, Usernames? Passwords? PPPoE? MAC Separation?
Is a BRAS or BGN functionally really needed or are these older concepts?
If CGNAT or DHCP is needed, what will host the CGNAT or DHCP service? The
core router, a linux box, or something else?
What about DNS?
Is a firewall needed in the core?
What else is needed?

Is there a guide out there somewhere? I know many cities are looking at
building their own network, and have similar questions. Access vendors are
willing to sell gear all day long, but then they leave it up to the
operator/city to answer these harder questions.

How would you build a access network from the ground up if you had the
resources and time to do so? Would you even use GPON? Even if GPON was not
used and another access technology like AE, VDSL2, or wireless was used I
think many of these questions would be the same.

Re: Muni Fiber and Politics

[Måns Nilsson]

Subject: Re: Muni Fiber and Politics Date: Wed, Jul 30, 2014 at 06:56:40PM 
-0500 Quoting Leo Bicknell (bickn...@ufp.org):
> 
> On Jul 30, 2014, at 1:47 AM, Mark Tinka  wrote:
> 
> > Symmetrical would be tough to do unless you're doing Active-
> > E.
> 
> I'm an outlier in my thinking, but I believe the best world would be
> where the muni offered L1 fiber, and leased access to it on a 
> non-discrimatory basis.  That would necessitate an Active-E solution
> since L1 would not have things like GPON splitters in it, but it 
> enables things like buying a dark fiber pair from your home to
> your business, and lighting it with your own optics.  That to me is
> a huge win.
> 
> It also means future upgrades are unencumbered.  Want to run 10GE?
> 100GE?  50x100GE WDM?  Please do.  You leased a dark fiber.  If the
> muni has "gear" (even just splitters) in the path they will gatekeeper
> upgrades.
> 
> It may be a smidge more expensive up front, but in the long run I
> think it will be cheaper, more reliable, and most importantly hugely
> more flexible.

GPON is basically unheard of in Sweden. All "fiber" access is either
copper to a switch in the basement/similar in multi-tenant houses or
direct pairs to CO. Some middle solutions exist where there's a rugged
switch in a pole or roadside cabinet, but they are exceptions. I think
the Amsterdam buildout is similar.

It is better, both for the customer and the provider. The only "loser"
is a potential third party acting as comms provider on L1, possibly L2.

-- 
Måns Nilsson primary/secondary/besserwisser/machina
MN-1334-RIPE +46 705 989668
DON'T go!!  I'm not HOWARD COSELL!!  I know POLISH JOKES ... WAIT!!
Don't go!!  I AM Howard Cosell! ... And I DON'T know Polish jokes!!


signature.asc
Description: Digital signature

Re: Owning a name

[Randy Bush]

> An update, apparently writs of attachment were sent for not only .ir, but
> also .sy and .kp ccTLDs as well, based on separate cases related to support
> for terrorism.

and they left out IL?

Re: Call for Nominations to ARIN AC and ARIN Board – Reminder

[Randy Bush]

i nominate pigasus, she'll fit right in