Re: Comcast DNS Contact

[Christopher Morrow]

On Mon, May 16, 2016 at 12:35 PM,  wrote:
>
> Can one of the Comcast DNS guru's contact me reference an issue with a
.gov resolution?
>
> Robert

​out of curiosity, is the .gov problem related to dnssec perhaps?​

Re: Question on peering strategies

[Jon Lewis]

On Mon, 16 May 2016, Reza Motamedi wrote:


Hi Nick,

Thanks for the reply.

Let me clarify another issue first, since I thought the colo's business
model is different at least in the US. So if AS-a puts its router in
Equinix, it should pay the same amount in the following two scenario (only
considering the interconnection cost and not the rent for racks and remote
hands and )?
1) AS-a only connects to the IX and establishes all inter-AS connections
through the IX.
2) AS-a connects to the IX, in addition to privately connecting to bunch of
other colo customers (these private connections can be either transit or
settlement-free peerings).
My understanding was that colos in the US charge per cross connect, so the
more you connect privately, the more you pay. This article may be old, but


Ports on the colo's IX, Equinix for example, will likely cost more than 
just a cross connect.  If you have peers with which you exchange enough 
traffic, it can make sense to remove that traffic from the IX and put it 
on PNI (cross connect) peering, leaving the IX port(s) for use primarily 
for peering with lots of "smaller peers" (in the amount of traffic 
exchanged).


Typically, if a peer is big enough to justify PNI, you won't want to 
fail-over to the IX as a backup, because doing so is likely to congest 
your or their IX links.  Of course, there are exceptions.  A PNI peer 
might not have enough ports to dedicate to PNI peering and might want to 
spread peering traffic over both PNI and IX evenly.


--
 Jon Lewis, MCP :)   |  I route
 |  therefore you are
_ http://www.lewis.org/~jlewis/pgp for PGP public key_

Re: Perspectives about customer M/A/C in triple play environments

[Spencer Ryan]

While it's possible I've never seen a mainstream ISP offering tripleplay
services (or even doubleplay) where the ATA isn't embedded in the CPE (eMTA
for DOCSIS)

As far as IPTV, at least the way UVerse does it the video traffic is all
untagged on the customer side, the gateway may try to do some QoS but this
allows you to plug a UVerse box in anywhere Ethernet works, along with
MoCA. This is simple, and kind of just works.


*Spencer Ryan* | Senior Systems Administrator | sr...@arbor.net
*Arbor Networks*
+1.734.794.5033 (d) | +1.734.846.2053 (m)
www.arbornetworks.com

On Mon, May 16, 2016 at 8:09 PM, Jason Lixfeld 
wrote:

> Hello,
>
> I think it’s fair to say that most broadband/FTTx customers don’t have to
> think very much or need to have a very high degree of understanding if they
> want to move their wired Internet device from one room or another in their
> house.
>
> Maybe to keep things simple, let’s assume that we’re talking about a
> relatively modern MDU unit where a customer has some sort of provider CPE
> in their in-suite telecom demark closet/box/what have you with some number
> of switched 'LAN’ ports on it, and each of those LAN ports would be wired
> to a wall jack somewhere.  Mr. or Ms. User can move their Internet device
> anywhere there is a wall jack and Bob’s your uncle.
>
> My question is around how this landscape changes in triple play
> environments.  As I understand it, most triple play deployments separate
> (in some cases VoIP,) TV and Internet traffic onto VLANs (Internet would be
> presented to the customer untagged).  The CPE would then allow the ISP to
> switch the video traffic onto a coax port, or maybe onto the CPE’s embedded
> switch, or maybe both.  For the sake of argument, let’s assume the provider
> is supplying an Ethernet based set-top-box, so customer should be able to
> connect the STB to any wall jack and it should just work.  And they should
> be able to connect their provider supplied ATA to any wall jack, and it
> should just work.  And they should be able to connect their Internet device
> to any wall jack and it should just work.
>
> Or should it?
>
> Are most CPEs that are provided by ISPs sophisticated enough to be able to
> put all service tags on all ports, and have those same ports act as
> untagged LAN ports as well?  If not, how do providers deal with this?  Do
> they dedicate one port for an IPTV STB?  One port for an ATA (assuming no
> built-in POTS on the CPE)?  And the rest of the ports for untagged
> Internet?  What if the customer has 2+ TVs?  Do they need to call in and
> have the provider remote in and provision another port for TV at the
> expense of some other service that might be running on that port already?
> Do they need to install a switch that does IGMP snooping?
>
> I feel like this all has the potential to become very complicated for the
> customer, and maybe the provider and their installers.  To me, the customer
> should continue to be dumb and unassuming.  They should be able to put
> whatever they want wherever they want and have it just work.  Is that how
> things actually are in the real world or are customers and providers making
> silent sacrifices for the sake of all this new fangled technology?
>

Re: Perspectives about customer M/A/C in triple play environments

[John Adams]

I have never seen this level of segmentation in any customer premises I
have worked on. Even in "triple-play" environments the handoff is nearly
always untagged ethernet and the downstream devices just work.

-j


On Mon, May 16, 2016 at 5:09 PM, Jason Lixfeld 
wrote:

> Hello,
>
> I think it’s fair to say that most broadband/FTTx customers don’t have to
> think very much or need to have a very high degree of understanding if they
> want to move their wired Internet device from one room or another in their
> house.
>
> Maybe to keep things simple, let’s assume that we’re talking about a
> relatively modern MDU unit where a customer has some sort of provider CPE
> in their in-suite telecom demark closet/box/what have you with some number
> of switched 'LAN’ ports on it, and each of those LAN ports would be wired
> to a wall jack somewhere.  Mr. or Ms. User can move their Internet device
> anywhere there is a wall jack and Bob’s your uncle.
>
> My question is around how this landscape changes in triple play
> environments.  As I understand it, most triple play deployments separate
> (in some cases VoIP,) TV and Internet traffic onto VLANs (Internet would be
> presented to the customer untagged).  The CPE would then allow the ISP to
> switch the video traffic onto a coax port, or maybe onto the CPE’s embedded
> switch, or maybe both.  For the sake of argument, let’s assume the provider
> is supplying an Ethernet based set-top-box, so customer should be able to
> connect the STB to any wall jack and it should just work.  And they should
> be able to connect their provider supplied ATA to any wall jack, and it
> should just work.  And they should be able to connect their Internet device
> to any wall jack and it should just work.
>
> Or should it?
>
> Are most CPEs that are provided by ISPs sophisticated enough to be able to
> put all service tags on all ports, and have those same ports act as
> untagged LAN ports as well?  If not, how do providers deal with this?  Do
> they dedicate one port for an IPTV STB?  One port for an ATA (assuming no
> built-in POTS on the CPE)?  And the rest of the ports for untagged
> Internet?  What if the customer has 2+ TVs?  Do they need to call in and
> have the provider remote in and provision another port for TV at the
> expense of some other service that might be running on that port already?
> Do they need to install a switch that does IGMP snooping?
>
> I feel like this all has the potential to become very complicated for the
> customer, and maybe the provider and their installers.  To me, the customer
> should continue to be dumb and unassuming.  They should be able to put
> whatever they want wherever they want and have it just work.  Is that how
> things actually are in the real world or are customers and providers making
> silent sacrifices for the sake of all this new fangled technology?
>

Re: Perspectives about customer M/A/C in triple play environments

[Jared Mauch]

Honestly if I'm paying for a TV service I should be able to plug in the device 
anywhere I have network. Look at what cell carriers have done with wifi 
calling. Have Internet? Have SMS, voice etc. 

If I'm consuming with a STB or a phone or VLC to ASCII plugin as long as I pass 
the authentication I should have access. 

Segmenting is just asking for trouble. 

Jared Mauch

> On May 16, 2016, at 8:09 PM, Jason Lixfeld  wrote:
> 
> They should be able to put whatever they want wherever they want and have it 
> just work.  Is that how things actually are in the real world or are 
> customers and providers making silent sacrifices for the sake of all this new 
> fangled technology?

Perspectives about customer M/A/C in triple play environments

[Jason Lixfeld]

Hello,

I think it’s fair to say that most broadband/FTTx customers don’t have to think 
very much or need to have a very high degree of understanding if they want to 
move their wired Internet device from one room or another in their house.  

Maybe to keep things simple, let’s assume that we’re talking about a relatively 
modern MDU unit where a customer has some sort of provider CPE in their 
in-suite telecom demark closet/box/what have you with some number of switched 
'LAN’ ports on it, and each of those LAN ports would be wired to a wall jack 
somewhere.  Mr. or Ms. User can move their Internet device anywhere there is a 
wall jack and Bob’s your uncle.

My question is around how this landscape changes in triple play environments.  
As I understand it, most triple play deployments separate (in some cases VoIP,) 
TV and Internet traffic onto VLANs (Internet would be presented to the customer 
untagged).  The CPE would then allow the ISP to switch the video traffic onto a 
coax port, or maybe onto the CPE’s embedded switch, or maybe both.  For the 
sake of argument, let’s assume the provider is supplying an Ethernet based 
set-top-box, so customer should be able to connect the STB to any wall jack and 
it should just work.  And they should be able to connect their provider 
supplied ATA to any wall jack, and it should just work.  And they should be 
able to connect their Internet device to any wall jack and it should just work.

Or should it?

Are most CPEs that are provided by ISPs sophisticated enough to be able to put 
all service tags on all ports, and have those same ports act as untagged LAN 
ports as well?  If not, how do providers deal with this?  Do they dedicate one 
port for an IPTV STB?  One port for an ATA (assuming no built-in POTS on the 
CPE)?  And the rest of the ports for untagged Internet?  What if the customer 
has 2+ TVs?  Do they need to call in and have the provider remote in and 
provision another port for TV at the expense of some other service that might 
be running on that port already?  Do they need to install a switch that does 
IGMP snooping?

I feel like this all has the potential to become very complicated for the 
customer, and maybe the provider and their installers.  To me, the customer 
should continue to be dumb and unassuming.  They should be able to put whatever 
they want wherever they want and have it just work.  Is that how things 
actually are in the real world or are customers and providers making silent 
sacrifices for the sake of all this new fangled technology?

Re: Level 3 issues?

[George Herbert]

Yes; you should subscribe to outa...@outages.org for better reports.  (Short 
summary - yes, no root cause/TTR yet).

George William Herbert
Sent from my iPhone

> On May 16, 2016, at 12:49 PM, David Hubbard  
> wrote:
> 
> Anyone seeing issues with Level 3 networking right now?  We’re seeing huge 
> latency and loss on traffic coming inbound (to us, AS33260) but it seems to 
> be at the peering points with other major ISP’s and Level 3.  Comcast for 
> example:
> 
>  333 ms21 ms70 ms  te-3-5-ur01.hershey.pa.pitt.comcast.net 
> [68.85.42.29]
>  4 *   33 ms   106 ms  162.151.48.173
>  5   214 ms54 ms41 ms  162.151.21.229
>  6   561 ms   764 ms   459 ms  4.68.71.133
> 
> Thanks,
> 
> David

Re: Level 3 issues?

[Nick Olsen]

Saw the same here. Legacy TW (AS4323) connection didn't take quite the hit that 
our Level3 (AS3356) did. But they were both seeing similar issues. Had to push 
traffic some specific routes toward cogent *shudder*.

   Nick Olsen
Sr. Network Operations  (855) FLSPEED  x106




 From: "David Hubbard" 
Sent: Monday, May 16, 2016 4:18 PM
To: "nanog@nanog.org" 
Subject: Re: Level 3 issues?
I just heard from someone there is suspicion that a fiber cut occurred in FL, 
possibly Miami area, and it has revealed a capacity issue on the L3 network. 
Haven't received official word on that yet, but I know our legacy TWTC 
connection is nearly as useless as our L3 connection thanks to the network 
merging activities.

David

On 5/16/16, 4:10 PM, "Jordan Medlen"  wrote:

>Have been seeing issues since just after 3P. Had to swing my traffic over to 
>another provider. Level3 says issues seen from Costa Rica on up to WDC.
>
>
>Thank you,
>
>Jordan Medlen
>Enterprise Communications Manager
>Bisk Education
>(813) 612-6207
>
> 
>
>On 5/16/16, 3:49 PM, "NANOG on behalf of David Hubbard" 
> wrote:
>
>>Anyone seeing issues with Level 3 networking right now? We're seeing huge 
>>latency and loss on traffic coming inbound (to us, AS33260) but it seems to 
>>be at the peering points with other major ISP's and Level 3. Comcast for 
>>example:
>>
>> 3 33 ms 21 ms 70 ms te-3-5-ur01.hershey.pa.pitt.comcast.net [68.85.42.29]
>> 4 * 33 ms 106 ms 162.151.48.173
>> 5 214 ms 54 ms 41 ms 162.151.21.229
>> 6 561 ms 764 ms 459 ms 4.68.71.133
>>
>>Thanks,
>>
>>David
>




Attachment 1
Description: Binary data

Re: Question on peering strategies

[Baldur Norddahl]

On 16 May 2016 at 22:06, Reza Motamedi  wrote:

> With respect to my second question, I was asking if is practical/reasonable
> to keep both the connection types to same network (say AS-b) at the same
> time, i.e., connect privately over a cross-connect and keep the public
> connection over the IX.
>


Router ports are expensive, so even if cross connects were free, you would
still use the public switch fabric until you reach a traffic level that
justifies a direct connection. The point of having a IX switch is that you
can connect to many others with just one single router port.

When you have the direct cross connect, you would not usually use the IX
switch in parallel for that AS. With the cross connect you have dedicated
bandwidth to the AS and you would want to reserve the IX switch port for
traffic to the remaining networks that you do not yet have a cross connect
to.

The cross connect is not a very good redundancy setup with regard to the IX
switch. Both usually go to the same router and share the same single point
of failure (your router is a single point of failure and the peer router is
a single point of failure). A cross connect is usual very reliable. You
would plan for your router to be down or the peer router to be down, and
have a backup path through some entirely geographic separate location.

In many cases your generic IP transit service is good enough backup. Your
direct peering is an optimization and if that is down, you go back to the
transit service.

Of course everyone are playing their own game and you might see anything
happening in the real world despite the above.

Regards,

Baldur

Re: Level 3 issues?

[David Hubbard]

I just heard from someone there is suspicion that a fiber cut occurred in FL, 
possibly Miami area, and it has revealed a capacity issue on the L3 network.  
Haven’t received official word on that yet, but I know our legacy TWTC 
connection is nearly as useless as our L3 connection thanks to the network 
merging activities.

David




On 5/16/16, 4:10 PM, "Jordan Medlen"  wrote:

>Have been seeing issues since just after 3P. Had to swing my traffic over to 
>another provider. Level3 says issues seen from Costa Rica on up to WDC.
>
>
>Thank you,
>
>Jordan Medlen
>Enterprise Communications Manager
>Bisk Education
>(813) 612-6207
> 
> 
>
>On 5/16/16, 3:49 PM, "NANOG on behalf of David Hubbard" 
> wrote:
>
>>Anyone seeing issues with Level 3 networking right now?  We’re seeing huge 
>>latency and loss on traffic coming inbound (to us, AS33260) but it seems to 
>>be at the peering points with other major ISP’s and Level 3.  Comcast for 
>>example:
>>
>>  333 ms21 ms70 ms  te-3-5-ur01.hershey.pa.pitt.comcast.net 
>> [68.85.42.29]
>>  4 *   33 ms   106 ms  162.151.48.173
>>  5   214 ms54 ms41 ms  162.151.21.229
>>  6   561 ms   764 ms   459 ms  4.68.71.133
>>
>>Thanks,
>>
>>David
>

Re: Level 3 issues?

[Jordan Medlen]

Have been seeing issues since just after 3P. Had to swing my traffic over to 
another provider. Level3 says issues seen from Costa Rica on up to WDC.


Thank you,

Jordan Medlen
Enterprise Communications Manager
Bisk Education
(813) 612-6207
 
 

On 5/16/16, 3:49 PM, "NANOG on behalf of David Hubbard" 
 wrote:

>Anyone seeing issues with Level 3 networking right now?  We’re seeing huge 
>latency and loss on traffic coming inbound (to us, AS33260) but it seems to be 
>at the peering points with other major ISP’s and Level 3.  Comcast for example:
>
>  333 ms21 ms70 ms  te-3-5-ur01.hershey.pa.pitt.comcast.net 
> [68.85.42.29]
>  4 *   33 ms   106 ms  162.151.48.173
>  5   214 ms54 ms41 ms  162.151.21.229
>  6   561 ms   764 ms   459 ms  4.68.71.133
>
>Thanks,
>
>David

RE: Level 3 issues?

[Ray Orsini]

I'm having trouble reaching my Birch connection from Verizon and another
Birch site. But no issues from FPL Fibernet which passes through Level3.

Regards,
Ray Orsini – CEO
Orsini IT, LLC – Technology Consultants
VOICE DATA  BANDWIDTH  SECURITY  SUPPORT
P: 305.967.6756 x1009   E: r...@orsiniit.com   TF: 844.OIT.VOIP
7900 NW 155th Street, Suite 103, Miami Lakes, FL 33016
http://www.orsiniit.com | View My Calendar | View/Pay Your Invoices | View
Your Tickets



-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of David Hubbard
Sent: Monday, May 16, 2016 3:49 PM
To: nanog@nanog.org
Subject: Level 3 issues?

Anyone seeing issues with Level 3 networking right now?  We’re seeing huge
latency and loss on traffic coming inbound (to us, AS33260) but it seems to
be at the peering points with other major ISP’s and Level 3.  Comcast for
example:

  333 ms21 ms70 ms  te-3-5-ur01.hershey.pa.pitt.comcast.net
[68.85.42.29]
  4 *   33 ms   106 ms  162.151.48.173
  5   214 ms54 ms41 ms  162.151.21.229
  6   561 ms   764 ms   459 ms  4.68.71.133

Thanks,

David

Re: Question on peering strategies

[Reza Motamedi]

Hi Nick,

Thanks for the reply.

Let me clarify another issue first, since I thought the colo's business
model is different at least in the US. So if AS-a puts its router in
Equinix, it should pay the same amount in the following two scenario (only
considering the interconnection cost and not the rent for racks and remote
hands and )?
1) AS-a only connects to the IX and establishes all inter-AS connections
through the IX.
2) AS-a connects to the IX, in addition to privately connecting to bunch of
other colo customers (these private connections can be either transit or
settlement-free peerings).
My understanding was that colos in the US charge per cross connect, so the
more you connect privately, the more you pay. This article may be old, but
I don't think much has changed:
https://www.telegeography.com/press/press-releases/2015/02/26/colocation-cross-connect-price-disparities-remain-between-u-s-europe/index.html

With respect to my second question, I was asking if is practical/reasonable
to keep both the connection types to same network (say AS-b) at the same
time, i.e., connect privately over a cross-connect and keep the public
connection over the IX.



Best Regards
Reza Motamedi (R.M)
Graduate Research Fellow
Oregon Network Research Group
Computer and Information Science
University of Oregon

On Mon, May 16, 2016 at 11:10 AM, Nick Ellermann  wrote:

> Reza,
> You maybe overthinking this one a bit. The economics are something to
> consider, however all public exchanges have different economics. With
> Equinix you pay pretty much a flat rate for a single 1Gbps/10Gbps link that
> includes the cost of facility cross-connect and public exchange access.  It
> is a nice one to many connection for all those various network and content
> networks your end users would appreciate direct connectivity. Depending on
> the public exchange you either have a single BGP session or a BGP session
> per network you are peering. Really after that, it's just BGP routing and
> route management. You do need to be careful about not being too overly
> dependent on a single public switch link, in some cases like at Equinix you
> may want multiple connections to redundant public exchange switches at that
> site. There is a balance you want to seek of number of paid upstream
> network transit providers you are connected to versus how many direct
> peering arrangements you have setup. It's not usually practical for a
> smaller network to have loads of BGP peers.  There are lots of good
> articles online about this fine balance and some good advice from
> experienced network operators.
>
> To your later questions. For your simple example, if AS-a and AS-b were
> both already on the public IX, and the link wasn't too overly critical then
> using the public IX switch maybe a good first step. However as that
> relationship matures, they most likely in a real world example may look to
> split the cost of the private cross-connect. If it was mutually beneficial.
> There is much more to public peering and transit than the technical
> conversation. Most of the larger networks on the public switches won't peer
> privately with anyone or only with extremely larger networks. To get a
> provider such as this to peer both privately and on the public exchange is
> not a technical issue, it's more of a business overhead and management
> issue.
> If you have a couple of quality upstream transit providers, they will be
> excellent failovers to a public switch outage.  Plan for the public switch
> to have as many problems as any upstream provider.
>
>
> Sincerely,
> Nick Ellermann – CTO & VP Cloud Services
> BroadAspect
>
> E: nellerm...@broadaspect.com
> P: 703-297-4639
> F: 703-996-4443
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is thus for use only by the intended recipient. If you
> received this in error, please contact the sender and delete the e-mail and
> its attachments from all computers.
>
> -Original Message-
> From: NANOG [mailto:nanog-bounces+nellermann=broadaspect@nanog.org]
> On Behalf Of Reza Motamedi
> Sent: Monday, May 16, 2016 1:46 PM
> To: nanog@nanog.org
> Subject: Question on peering strategies
>
> Dear Nanogers,
>
> I have a question about common/best network interconnection practices.
> Assume that two networks (let's refer to them as AS-a and AS-b) are
> present in a colocation facility say Equinix LA. As many of you know,
> Equininx runs an IXP in LA as well. So AS-as and AS-b can interconnct
> 1) using private cross-connect
> 2) through the public IXP's switching fabric.
> Is it a common/good practice for the two networks to establish connections
> both through the IXP and also using a private cross-connect?
>
> I was thinking considering the cost of cross-connects (my understanding is
> that the colocation provider charges the customers for each cross-connect
> in addition to the rent of the rack or cage or whatever), it would 

RE: Cost-effectivenesss of highly-accurate clocks for NTP

[Jameson, Daniel]

Give this guy a look,  1RMU form factor, GPS with Rubidium holdover (7 days) if 
you need it very inexpensive,  
http://www.fibrolan.com/FibroLAN/SendFile.asp?DBID=1=1=3979
or
http://www.fibrolan.com/FibroLAN/SendFile.asp?DBID=1=1=3978  if you 
have a SYC-E source
or  If you just need highly accurate NTP without the holdover,  you can do it 
with FBSD,  and a GPS device.
http://www.satsignal.eu/ntp/FreeBSD-GPS-PPS.htm
http://www.brandywinecomm.com/46-products/bus-level-plug-in-boards/212-pcie-1588-universal-timing-card



-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Mel Beckman
Sent: Monday, May 16, 2016 11:27 AM
To: Lamar Owen
Cc: nanog@nanog.org
Subject: Re: Cost-effectivenesss of highly-accurate clocks for NTP

Lamar,

Although VoIP has different technical challenges than TDM, they are all 
surmountable. Usually VoIP problems result from poor network design (e.g., 
mixed traffic with no QoS, Internet transport with no guarantees, etc). Public 
safety networks are generally well designed, don’t use the Internet, and 
support a single type of traffic: audio streams. I’ve done demonstrations of 
VoIP that works well in this environment, and you get the advantages of IP 
routing for redundancy, as opposed to clunky T1 failover mechanisms usually 
based on hardware switches.

But public safety customers are not swayed. TDM works, and it’s the gold 
standard. They don’t want to change, and you can’t make them. They only see 
risk, no reward. BTW, in the TDM model, remote data and audio are usually two 
different systems, which is probably a good idea for redundancy: you might lose 
audio or data, but rarely both.

In any event, I’m only proposing that public safety upgrade their audio systems 
to VoIP (or cellular-style private GSM, which is in essence VoIP). But nobody 
is willing.

 -mel

> On May 16, 2016, at 9:02 AM, Lamar Owen  wrote:
> 
> On 05/15/2016 01:05 PM, Eric S. Raymond wrote:
>> I'm not used to thinking of IT as a relatively low-challenge environment! 
> 
> I actually changed careers from broadcast engineering to IT to lower my 
> stress level and 'personal bandwidth challenge.'  And, yes, it worked.  In my 
> case, I'm doing IT for radio and optical astronomy, and at least the timing 
> aspect is higher-challenge that most IT environments.
> 
>> You're implicitly suggesting there might be a technical case for replacing 
>> these T1/T3 trunks with some kind of VOIP provisioning less dependent on 
>> accurate time synch. Do you think that's true? 
> 
> While I know the question was directed at Mel specifically, I'll just say 
> from the point of view of a T1 voice trunk customer that I hope to never see 
> it go to a VoIP solution.  VoIP codecs can have some serious latency issues; 
> I already notice the round-trip delay if I try to carry on a conversation 
> between our internal VoIP system and someone on a cell phone.  And this is 
> before codec artifacting (and cascaded codec scrambling) is counted.  Can we 
> please keep straight μ-law (A-law if relevant) lossless DS0 PCM timeslices 
> for trunklines so we get at least one less lossy codec cascade?  Or have you 
> never experimented with what happens when you cascade G.722 with G.729 with 
> G.726 and then G.711 and back?  Calls become mangled gibberish.
> 
> I would find it interesting to see how many carriers are still doing large 
> amounts of SONET, as that is the biggest use-case for high-stability timing.
> 

Level 3 issues?

[David Hubbard]

Anyone seeing issues with Level 3 networking right now?  We’re seeing huge 
latency and loss on traffic coming inbound (to us, AS33260) but it seems to be 
at the peering points with other major ISP’s and Level 3.  Comcast for example:

  333 ms21 ms70 ms  te-3-5-ur01.hershey.pa.pitt.comcast.net 
[68.85.42.29]
  4 *   33 ms   106 ms  162.151.48.173
  5   214 ms54 ms41 ms  162.151.21.229
  6   561 ms   764 ms   459 ms  4.68.71.133

Thanks,

David

RE: Question on peering strategies

[Nick Ellermann]

Reza, 
You maybe overthinking this one a bit. The economics are something to consider, 
however all public exchanges have different economics. With Equinix you pay 
pretty much a flat rate for a single 1Gbps/10Gbps link that includes the cost 
of facility cross-connect and public exchange access.  It is a nice one to many 
connection for all those various network and content networks your end users 
would appreciate direct connectivity. Depending on the public exchange you 
either have a single BGP session or a BGP session per network you are peering. 
Really after that, it's just BGP routing and route management. You do need to 
be careful about not being too overly dependent on a single public switch link, 
in some cases like at Equinix you may want multiple connections to redundant 
public exchange switches at that site. There is a balance you want to seek of 
number of paid upstream network transit providers you are connected to versus 
how many direct peering arrangements you have setup. It's not usually practical 
for a smaller network to have loads of BGP peers.  There are lots of good 
articles online about this fine balance and some good advice from experienced 
network operators. 

To your later questions. For your simple example, if AS-a and AS-b were both 
already on the public IX, and the link wasn't too overly critical then using 
the public IX switch maybe a good first step. However as that relationship 
matures, they most likely in a real world example may look to split the cost of 
the private cross-connect. If it was mutually beneficial. There is much more to 
public peering and transit than the technical conversation. Most of the larger 
networks on the public switches won't peer privately with anyone or only with 
extremely larger networks. To get a provider such as this to peer both 
privately and on the public exchange is not a technical issue, it's more of a 
business overhead and management issue. 
If you have a couple of quality upstream transit providers, they will be 
excellent failovers to a public switch outage.  Plan for the public switch to 
have as many problems as any upstream provider. 


Sincerely,
Nick Ellermann – CTO & VP Cloud Services
BroadAspect
 
E: nellerm...@broadaspect.com 
P: 703-297-4639
F: 703-996-4443
 
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-Original Message-
From: NANOG [mailto:nanog-bounces+nellermann=broadaspect@nanog.org] On 
Behalf Of Reza Motamedi
Sent: Monday, May 16, 2016 1:46 PM
To: nanog@nanog.org
Subject: Question on peering strategies

Dear Nanogers,

I have a question about common/best network interconnection practices.
Assume that two networks (let's refer to them as AS-a and AS-b) are present in 
a colocation facility say Equinix LA. As many of you know, Equininx runs an IXP 
in LA as well. So AS-as and AS-b can interconnct
1) using private cross-connect
2) through the public IXP's switching fabric.
Is it a common/good practice for the two networks to establish connections both 
through the IXP and also using a private cross-connect?

I was thinking considering the cost of cross-connects (my understanding is that 
the colocation provider charges the customers for each cross-connect in 
addition to the rent of the rack or cage or whatever), it would not be 
economically reasonable to have both. Although, if the cross-connect is the 
primary method of interconnection, and the IXP provides a router-server the 
public-peering over IXP would essentially be free. So it might makes sense to 
assume that for the private cross-connect, there exists a back-up connection 
though the IXP. Anyway, I guess some discussion may give more insight about 
which one is more reasonable to assume and do.

Now my last question is that if the two connections exist (one private 
cross-connect and another back-up through the IXP), what are the chances that 
periodically launched traceroutes that pass the inter-AS connection in that 
colo see both types of connection in a week. I guess what I'm asking is how 
often back-up routes are taken? Can the networks do load balancing on the two 
connection and essentially use them as primary routes?

Best Regards
Reza Motamedi (R.M)
Graduate Research Fellow
Oregon Network Research Group
Computer and Information Science
University of Oregon

Question on peering strategies

[Reza Motamedi]

Dear Nanogers,

I have a question about common/best network interconnection practices.
Assume that two networks (let's refer to them as AS-a and AS-b) are present
in a colocation facility say Equinix LA. As many of you know, Equininx runs
an IXP in LA as well. So AS-as and AS-b can interconnct
1) using private cross-connect
2) through the public IXP's switching fabric.
Is it a common/good practice for the two networks to establish connections
both through the IXP and also using a private cross-connect?

I was thinking considering the cost of cross-connects (my understanding is
that the colocation provider charges the customers for each cross-connect
in addition to the rent of the rack or cage or whatever), it would not be
economically reasonable to have both. Although, if the cross-connect is the
primary method of interconnection, and the IXP provides a router-server the
public-peering over IXP would essentially be free. So it might makes sense
to assume that for the private cross-connect, there exists a back-up
connection though the IXP. Anyway, I guess some discussion may give more
insight about which one is more reasonable to assume and do.

Now my last question is that if the two connections exist (one private
cross-connect and another back-up through the IXP), what are the chances
that periodically launched traceroutes that pass the inter-AS connection in
that colo see both types of connection in a week. I guess what I'm asking
is how often back-up routes are taken? Can the networks do load balancing
on the two connection and essentially use them as primary routes?

Best Regards
Reza Motamedi (R.M)
Graduate Research Fellow
Oregon Network Research Group
Computer and Information Science
University of Oregon

Comcast DNS Contact

[rwebb]

Can one of the Comcast DNS guru's contact me reference an issue with a .gov 
resolution?


Robert

Re: Cost-effectivenesss of highly-accurate clocks for NTP

[Mel Beckman]

Lamar,

Although VoIP has different technical challenges than TDM, they are all 
surmountable. Usually VoIP problems result from poor network design (e.g., 
mixed traffic with no QoS, Internet transport with no guarantees, etc). Public 
safety networks are generally well designed, don’t use the Internet, and 
support a single type of traffic: audio streams. I’ve done demonstrations of 
VoIP that works well in this environment, and you get the advantages of IP 
routing for redundancy, as opposed to clunky T1 failover mechanisms usually 
based on hardware switches.

But public safety customers are not swayed. TDM works, and it’s the gold 
standard. They don’t want to change, and you can’t make them. They only see 
risk, no reward. BTW, in the TDM model, remote data and audio are usually two 
different systems, which is probably a good idea for redundancy: you might lose 
audio or data, but rarely both.

In any event, I’m only proposing that public safety upgrade their audio systems 
to VoIP (or cellular-style private GSM, which is in essence VoIP). But nobody 
is willing.

 -mel

> On May 16, 2016, at 9:02 AM, Lamar Owen  wrote:
> 
> On 05/15/2016 01:05 PM, Eric S. Raymond wrote:
>> I'm not used to thinking of IT as a relatively low-challenge environment! 
> 
> I actually changed careers from broadcast engineering to IT to lower my 
> stress level and 'personal bandwidth challenge.'  And, yes, it worked.  In my 
> case, I'm doing IT for radio and optical astronomy, and at least the timing 
> aspect is higher-challenge that most IT environments.
> 
>> You're implicitly suggesting there might be a technical case for replacing 
>> these T1/T3 trunks with some kind of VOIP provisioning less dependent on 
>> accurate time synch. Do you think that's true? 
> 
> While I know the question was directed at Mel specifically, I'll just say 
> from the point of view of a T1 voice trunk customer that I hope to never see 
> it go to a VoIP solution.  VoIP codecs can have some serious latency issues; 
> I already notice the round-trip delay if I try to carry on a conversation 
> between our internal VoIP system and someone on a cell phone.  And this is 
> before codec artifacting (and cascaded codec scrambling) is counted.  Can we 
> please keep straight μ-law (A-law if relevant) lossless DS0 PCM timeslices 
> for trunklines so we get at least one less lossy codec cascade?  Or have you 
> never experimented with what happens when you cascade G.722 with G.729 with 
> G.726 and then G.711 and back?  Calls become mangled gibberish.
> 
> I would find it interesting to see how many carriers are still doing large 
> amounts of SONET, as that is the biggest use-case for high-stability timing.
> 

Re: Cost-effectivenesss of highly-accurate clocks for NTP

[Lamar Owen]

On 05/15/2016 03:16 PM, Måns Nilsson wrote:
...If you think the IP implementations in IoT devices are naîve, wait 
until you've seen what passes for broadcast quality network 
engineering. Shoving digital audio samples in raw Ethernet frames is 
at least 20 years old, but the last perhaps 5 years has seen some 
progress in actually using IP to carry audio streams. (this is 
close-to-realtime audio, not file transfers, btw.) 


Close to realtime is a true statement.  Using an IP STL 
(studio-transmitter link) has enough latency that the announcer can no 
longer use the air signal as a monitor.


And the security side of things is a pretty serious issue; just ask a 
major IP STL appliance vendor about the recent hijacking of some of 
their customers' IP STL devices yeah, a random intruder on the 
internet hijacked several radio stations' IP STL's and began 
broadcasting their content over the radio.  Not pretty.  I advise any of 
my remaining broadcast clients that if they are going to an IP STL that 
they put in a dedicated point to point IP link without publicly routable 
IP addresses.


Digital audio for broadcast STL's is old tech; we were doing G.722/G.723 
over switched-56 in the early 90's.  But using a public-facing internet 
connection with no firewalling for an IP STL appliance like the Barix 
boxes and the Tieline boxes and similar? That borders on networking 
malpractice.


... But, to try to return to "relevant for NANOG", there are actual 
products requiring microsecond precision being sold. And used. And 
we've found that those products don't have a very good holdover. ... 

Television broadcast is another excellent example of timing needs; thanks.

Valdis mentioned the scariest thing the scariest thing I've seen 
recently?  Windows NT 3.5 being used for a transmitter control system, 
within the past five years.  I will agree with Valdis on the scary 
aspects of the public safety communications Mel mentioned. Thanks, Mel, 
for the educational post.

Re: Cost-effectivenesss of highly-accurate clocks for NTP

[Lamar Owen]

On 05/15/2016 01:05 PM, Eric S. Raymond wrote:
I'm not used to thinking of IT as a relatively low-challenge environment! 


I actually changed careers from broadcast engineering to IT to lower my 
stress level and 'personal bandwidth challenge.'  And, yes, it worked.  
In my case, I'm doing IT for radio and optical astronomy, and at least 
the timing aspect is higher-challenge that most IT environments.


You're implicitly suggesting there might be a technical case for 
replacing these T1/T3 trunks with some kind of VOIP provisioning less 
dependent on accurate time synch. Do you think that's true? 


While I know the question was directed at Mel specifically, I'll just 
say from the point of view of a T1 voice trunk customer that I hope to 
never see it go to a VoIP solution.  VoIP codecs can have some serious 
latency issues; I already notice the round-trip delay if I try to carry 
on a conversation between our internal VoIP system and someone on a cell 
phone.  And this is before codec artifacting (and cascaded codec 
scrambling) is counted.  Can we please keep straight μ-law (A-law if 
relevant) lossless DS0 PCM timeslices for trunklines so we get at least 
one less lossy codec cascade?  Or have you never experimented with what 
happens when you cascade G.722 with G.729 with G.726 and then G.711 and 
back?  Calls become mangled gibberish.


I would find it interesting to see how many carriers are still doing 
large amounts of SONET, as that is the biggest use-case for 
high-stability timing.

Re: Cost-effectivenesss of highly-accurate clocks for NTP

[sthaug]

> I was just thing about this WAN jitter issue myself.  I'm wondering how many
> folks put NTP traffic in priority queues?  At least for devices in your
> managed IP ranges.  Seems like that would improve jitter.  Would like to
> hear about others doing this successfully prior to suggesting it for our
> network.

NTP, no. Not worth it. PTP, synched to a suitably accurate source,
absolutely.

Steinar Haug, AS2116

RE: Cost-effectivenesss of highly-accurate clocks for NTP

[Chuck Church]

-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Leo Bicknell
Sent: Monday, May 16, 2016 10:28 AM
To: nanog@nanog.org
Subject: Re: Cost-effectivenesss of highly-accurate clocks for NTP

>For a typical site, there are two distinct desires from the same NTP
process.

>First, synchronization with the rest of the world, which is generally over
the WAN and the topic that was well discussed in your post.
>I agree completely that the largest factor is WAN jitter.

>Leo Bicknell - bickn...@ufp.org
>PGP keys at http://www.ufp.org/~bicknell/

I was just thing about this WAN jitter issue myself.  I'm wondering how many
folks put NTP traffic in priority queues?  At least for devices in your
managed IP ranges.  Seems like that would improve jitter.  Would like to
hear about others doing this successfully prior to suggesting it for our
network.

Chuck

Re: Cost-effectivenesss of highly-accurate clocks for NTP

[Leo Bicknell]

In a message written on Fri, May 13, 2016 at 03:39:27PM -0400, Eric S. Raymond 
wrote:
> According to RFC 5095 expected accuracy of NTP time is "several tens
> of milliseconds." User expectations seem to evolved to on the close
> order of 10ms.  I think it's not by coincidence this is pretty close
> to the jitter in ping times I see when I bounce ICMP off a
> well-provisioned site like (say) google.com through my Verizon FIOS
> connection.

For a typical site, there are two distinct desires from the same
NTP process.

First, syncronization with the rest of the world, which is generally
over the WAN and the topic that was well discussed in your post.
I agree completely that the largest factor is WAN jitter.

The other is local syncronization, insuring multiple servers have
the same time for log analysis purposes and such.  This typically
takes place across a lightly loaded ethernet fabric, perhaps with
small latency (across a compus).  Jitter here is much, much smaller.

Does the limitation of accuracy remain jitter in the low jitter
LAN/Campus enviornment, or does that expose other issues like the
quality of oscellators, OS scheduling, etc?  Or perhaps another
way, is it possible to get say 10's or 100's of nanosecond accuracy
in the lan/campus?

-- 
Leo Bicknell - bickn...@ufp.org
PGP keys at http://www.ufp.org/~bicknell/


pgpE9K0yZ7Yjy.pgp
Description: PGP signature