Re: [EXTERNAL] Re: VoIP Provider DDoSes

[K. Scott Helms]

The problem with this approach, and with scrubbing centers more generally,
is that while the cure might be better than the disease it doesn't result
in usable VOIP.  Voice customers don't care if things are _better_ but
their MOS scores are still below 2.

Scott Helms



On Wed, Sep 22, 2021 at 11:58 AM Compton, Rich A 
wrote:

> FYI, UTRS (Unwanted Traffic Removal Service
> https://team-cymru.com/community-services/utrs/) from Team Cymru is a
> free service where you can send a blackhole advertisement (sacrificing the
> one IP that’s under attack to save the rest of the network) and they will
> propagate that via BGP to hundreds of other ASNs which will then blackhole
> traffic to that IP.  This can drastically reduce the amount of DDoS traffic
> that is received by the victim network.
>
>
>
> -Rich
>
>
>
> *From: *NANOG  on
> behalf of Mike Hammett 
> *Date: *Wednesday, September 22, 2021 at 9:29 AM
> *To: *Terrance Devor 
> *Cc: *NANOG list 
> *Subject: *[EXTERNAL] Re: VoIP Provider DDoSes
>
>
>
> *CAUTION:* The e-mail below is from an external source. Please exercise
> caution before opening attachments, clicking links, or following guidance.
>
> Fail2Ban on a couple of dozen servers may not be sufficient to address 400
> gigs of traffic.
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
>
> --
>
> *From: *"Terrance Devor" 
> *To: *"Mike Hammett" 
> *Cc: *"NANOG" 
> *Sent: *Wednesday, September 22, 2021 10:24:07 AM
> *Subject: *Re: VoIP Provider DDoSes
>
> Fail2Ban and give ourselves a pat on the back..
>
>
>
> On Wed, Sep 22, 2021 at 9:12 AM Mike Hammett  wrote:
>
> https://twit.tv/shows/security-now/episodes/837?autostart=false
>
>
>
>
>
> It looks like Security Now covered this yesterday. They claimed that,
> "There  is  currently  no  provider of  large  pipe  VoIP  protocol  DDoS
>  protection."
>
>
>
> Are any of the cloud DDoS mitigation services offering a service like this.
> --
>
> *From: *"Mike Hammett" 
> *To: *"NANOG" 
> *Sent: *Tuesday, September 21, 2021 4:19:42 PM
> *Subject: *VoIP Provider DDoSes
>
> As many may know, a particular VoIP supplier is suffering a DDoS.
> https://twitter.com/voipms
>
>
>
> Are your garden variety DDoS mitigation platforms or services equipped to
> handle DDoSes of VoIP services? What nuances does one have to be cognizant
> of? A WAF doesn't mean much to SIP, IAX2, RTP, etc.
>
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
>
>
>
> The contents of this e-mail message and
> any attachments are intended solely for the
> addressee(s) and may contain confidential
> and/or legally privileged information. If you
> are not the intended recipient of this message
> or if this message has been addressed to you
> in error, please immediately alert the sender
> by reply e-mail and then delete this message
> and any attachments. If you are not the
> intended recipient, you are notified that
> any use, dissemination, distribution, copying,
> or storage of this message or any attachment
> is strictly prohibited.
>

Re: EXTERNAL: Re: VoIP Provider DDoSes

[K. Scott Helms]

I'm going to be reaching out to both of the organizations you listed, but I
don't see any of their documentation mentioning SIP, RTP, or any of the
"normal" VOIP protocols or use cases.

Scott Helms



On Wed, Sep 22, 2021 at 9:18 AM Ray Orsini  wrote:

> Yes there are. I was about to message Steve about the correction. Corero
> and path.net are options. There are others.
> [image: OIT Website] 
> Ray Orsini​
> Chief Executive Officer
> OIT, LLC
>  *305.967.6756 x1009* <305.967.6756%20x1009>  |   *305.571.6272*
>  *r...@oit.co*   |  [image: https://www.oit.co]
>  * www.oit.co* 
>  oit.co/ray
> [image: Facebook] 
> [image: LinkedIn] 
> [image: Twitter] 
> [image: YouTube] 
>
> *How are we doing? We'd love to hear your feedback. https://go.oit.co/review*
> 
> --
> *From:* NANOG  on behalf of Mike
> Hammett 
> *Sent:* Wednesday, September 22, 2021 9:08:22 AM
> *To:* NANOG 
> *Subject:* EXTERNAL: Re: VoIP Provider DDoSes
>
>
> CAUTION: This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe. If you are unsure, please forward this email to the
> CSE team for review.
>
> https://twit.tv/shows/security-now/episodes/837?autostart=false
>
>
> It looks like Security Now covered this yesterday. They claimed that,
> "There  is  currently  no  provider of  large  pipe  VoIP  protocol  DDoS
>  protection."
>
> Are any of the cloud DDoS mitigation services offering a service like this.
>
> --
> *From: *"Mike Hammett" 
> *To: *"NANOG" 
> *Sent: *Tuesday, September 21, 2021 4:19:42 PM
> *Subject: *VoIP Provider DDoSes
>
> As many may know, a particular VoIP supplier is suffering a DDoS.
> https://twitter.com/voipms
>
> Are your garden variety DDoS mitigation platforms or services equipped to
> handle DDoSes of VoIP services? What nuances does one have to be cognizant
> of? A WAF doesn't mean much to SIP, IAX2, RTP, etc.
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
>

Re: SITR/SHAKEN implementation in effect today (June 30 2021)

[K. Scott Helms]

On Fri, Jul 9, 2021 at 4:47 PM Michael Thomas  wrote:

>
> On 7/9/21 1:36 PM, K. Scott Helms wrote:
> > Nothing will change immediately.  Having said that, I do expect that
> > we will see much more effective enforcement. The investigations will
> > come from the ITG (Industry Traceback Group) with enforcement
> > coming from FCC or FTC depending on the actual offense.  The problem
> > has been that it's been far too easy for robocalling companies to hop
> > from one telecom provider to another.  Now there are requirements
> > around "know your customer" that telecom operators have to follow and
> > the ITG will have a much better chance of figuring out who the bad
> > actor is than they have in the past.
>
> The thing is that that shouldn't have been held up by rolling out STIR.
> With email, there was nothing akin to the FCC so it was really the only
> name-and-shame stick we had. This could have been done years ago.
>

It wouldn't work the same and I say that as someone who ran email for ISPs
for decades and just got done with a STIR/SHAKEN implementation.  There's
far more money in robocalls than there ever has been in spam.  The other
thing is that the technologies are fundamentally different.  Don't get me
wrong, there are parallels, but comparing email to the various flavors of
telephony (POTS, SIP, MGCP, H.323, etc) isn't all that useful because
they're so different.  When I get an email as a provider I can always
figure out the path it took to get to me.  The same is not at all true for
a call, though I can trace it to a degree via the CDRs from carriers I work
with.  Much of the call path will be opaque and often in the case of
robocallers it's intentionally so.  Number porting is another (big)
difference.  We could always choose to forward email for a customer who
left our service, but imagine if email literally let that person take their
address with them regardless of who was providing the hosting for the email.



> > Longer term I worry that this will lead to more attacks on PBXs,
> > eSBCs, and VOIP handsets to be able to call either from that endpoint
> > itself or be able to use the SIP credentials. The market for robocalls
> > will certainly not disappear.
> >
> A meta question that really needs to be asked these days is why we even
> need telco telephony anymore. A lot of problems go away if you are not
> in thrall to 100 year old technology and its accreted kruft.
>

Robocalls really aren't a product of the legacy PSTN.  Today almost none of
them originate from anywhere but VOIP.  Now, you can certainly say that if
SS7 had robust authentication mechanisms that we could then trust caller ID
(more) but there's no sign of us abandoning the PSTN anytime soon.  Having
said that, there's any number of protocols we rely on today that have the
exact same gap.  BGP is arguably even worse than SS7.

tl;dr You can no more get rid of telephone companies than you can get rid
of ISPs.



>
> Mike
>
>

Re: SITR/SHAKEN implementation in effect today (June 30 2021)

[K. Scott Helms]

Nothing will change immediately.  Having said that, I do expect that we
will see much more effective enforcement.  The investigations will come
from the ITG (Industry Traceback Group) with enforcement coming from FCC or
FTC depending on the actual offense.  The problem has been that it's been
far too easy for robocalling companies to hop from one telecom provider to
another.  Now there are requirements around "know your customer" that
telecom operators have to follow and the ITG will have a much better chance
of figuring out who the bad actor is than they have in the past.

Longer term I worry that this will lead to more attacks on PBXs, eSBCs, and
VOIP handsets to be able to call either from that endpoint itself or be
able to use the SIP credentials.  The market for robocalls will certainly
not disappear.

Scott Helms



On Fri, Jul 9, 2021 at 1:07 PM Michael Thomas  wrote:

> Nothing has changed for me either. Color me surprised. The real proof will
> be to see if the originating domain can be determined, and whether the
> receiving domain does anything about it.
>
> Mike
> On 7/9/21 9:42 AM, Brandon Svec via NANOG wrote:
>
> I’m getting the same or more, but did anyone really expect they would stop
> July 1? It will take time for complaints to be tracked down and operators
> to take actions, right?
>
> Brandon
>
> On Fri, Jul 9, 2021 at 6:49 AM Josh Luthman 
> wrote:
>
>> Subjectively speaking, I'm still getting the same amount of spam phone
>> calls.
>>
>> I'm certainly getting a lot more spam SMS to my cell.  Almost all of them
>> in my entire life starting July 1...
>>
>>
>> Josh Luthman
>> 24/7 Help Desk: 937-552-2340
>> Direct: 937-552-2343
>> 1100 Wayne St
>> 
>> Suite 1337
>> 
>> Troy, OH 45373
>> 
>>
>>
>> On Fri, Jul 9, 2021 at 9:40 AM Jeff Shultz 
>> wrote:
>>
>>> All I know is that I am getting a lot fewer bogus calls on my cell phone
>>> than I was this time last month.
>>>
>>> On Fri, Jul 9, 2021, 06:17 Ryan Finnesey via NANOG 
>>> wrote:
>>>
 This should help with Robo calls a lot.

 -Original Message-
 From: NANOG  On
 Behalf Of Sean Donelan
 Sent: Wednesday, June 30, 2021 2:31 PM
 To: nanog@nanog.org
 Subject: SITR/SHAKEN implementation in effect today (June 30 2021)


 STIR/SHAKEN Broadly Implemented Starting Today
 https://www.fcc.gov/document/stirshaken-broadly-implemented-starting-today

 WASHINGTON, June 30, 2021—FCC Acting Chairwoman Jessica Rosenworcel
 today announced that the largest voice service providers are now using
 STIR/SHAKEN caller ID authentication standards in their IP networks, in
 accordance with the deadline set by the FCC. This widespread implementation
 helps protect consumers against malicious spoofed robocalls and helps law
 enforcement track bad actors. The STIR/SHAKEN standards serve as a common
 digital language used by phone networks, allowing valid information to pass
 from provider to provider which, among other things, informs blocking tools
 of possible suspicious calls.
>>>
>>> --
> Brandon Svec
> 15106862204 ☎️ or 
>
>

Re: Email and Web Hosting

[K. Scott Helms]

Two decent options, one on prem and the other fully hosted.

Tucows/OpenSRS has a fully hosted email offering that was built for ISPs to
resell.  (They also have domain registration and some other ISP focused
services.)

https://opensrs.com/services/hosted-email/

MagicMail is an email (including webmail) suite that you run on prem.  It
is comparatively inexpensive but also fully supported.  It's built largely
on qmail, but they replaced some of the components to deal with spam and
virus filtering more efficiently.

https://www.magicmail.com/

I have direct experience with both and have used them both for ISPs
specifically.

Scott Helms



On Tue, Jul 6, 2021 at 10:44 AM Steve Saner  wrote:

> I hope this isn't too far off topic for this list.
>
> We acquired a small ISP a couple years ago that has its roots in the
> "local ISPs" of the 90s. This ISP is still hosting email and web services
> for customers both on company domains as well as customer domains. There is
> some decent revenue coming from these services, but cost of maintenance is
> becoming a challenge. We are looking at migrating to another platform or
> completely discontinuing those services.
>
> I'm wondering if others here have gone through that process and have any
> advice as to how to go about it.
>
> --
>
> Steve Saner | Senior Network Engineer
>
> ideatek INTERNET FREEDOM FOR ALL
>
> Cell: 620-860-9433 | 111 Old Mill Lane, Buhler, KS 67522 | ideatek.com
> 
>
> This email transmission and any documents, files or previous email
> messages attached to it may contain confidential information. If the reader
> of this message is not the intended recipient or the employee or agent
> responsible for delivering the message to the intended recipient, you are
> hereby notified that any dissemination, distribution or copying of this
> communication is strictly prohibited. If you are not or believe you may not
> be the intended recipient, please advise the sender immediately by return
> email or by calling 620.543.5026. Then, please take all steps necessary to
> permanently delete the email and all attachments from your computer system.
> No trees were affected by this transmission – though a few billion photons
> were mildly inconvenienced.
>

Re: Google uploading your plain text passwords

[K. Scott Helms]

Bill,

It's not a theory and it doesn't have to be Chrome to work.  Javascript
does the work to decrypt the data and it's not browser specific.

Read the PDF I supplied that details_excatly_ how the key exchange and
encryption works.

Scott Helms



On Sat, Jun 12, 2021 at 10:35 PM William Herrin  wrote:

> On Sat, Jun 12, 2021 at 3:55 PM K. Scott Helms 
> wrote:
> > I don't think you're lying, but you are mistaken.
> >
> > "I'm not lying. Google's server at passwords.google.com
> > composed an html web page containing my plaintext passwords and sent
> > it to me. Not decrypted by my browser after combining it with a
> > locally stored key. "
> >
> > So, you're not describing all of the possible ways to decrypt data.
> What's happening is that the keys to decrypt the passwords are handed to
> your client (with some checks like a local admin password or pin) when you
> attempt to decrypt a given password.  The passwords _are_ decrypted on your
> device and you did not get a HTML page with your passwords.  Please, go
> look at the source yourself.  What you got was a page that's almost
> entirely javascript and that includes the functions that handle the
> decryption.
> >
> > Don't take my word for it, "When you log in to a website while signed in
> to Chrome, Chrome encrypts your username and password with a secret key
> known only to your device. Then it sends an obscured copy of your data to
> Google. Because the encryption happens before Google’s servers get the
> information, nobody, including Google, learns your username or password."
>
> There's a problem with your theory. The browser I viewed the passwords
> from Google in wasn't Chrome. And it didn't have a local copy of any
> Google passwords or keys. The only place they could have come from was
> Google's server.
>
> Regards,
> Bill Herrin
>
>
>
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/
>

Re: Google uploading your plain text passwords

[K. Scott Helms]

Bill,

I don't think you're lying, but you are mistaken.

"I'm not lying. Google's server at passwords.google.com
composed an html web page containing my plaintext passwords and sent
it to me. Not decrypted by my browser after combining it with a
locally stored key. "

So, you're not describing all of the possible ways to decrypt data.  What's
happening is that the keys to decrypt the passwords are handed to your
client (with some checks like a local admin password or pin) when you
attempt to decrypt a given password.  The passwords _are_ decrypted on your
device and you did not get a HTML page with your passwords.  Please, go
look at the source yourself.  What you got was a page that's almost
entirely javascript and that includes the functions that handle the
decryption.

Don't take my word for it, "When you log in to a website while signed in to
Chrome, Chrome encrypts your username and password with a secret key known
only to your device. Then it sends an obscured copy of your data to Google.
Because the encryption happens before Google’s servers get the information,
nobody, including Google, learns your username or password."

https://support.google.com/chrome/answer/10311524?hl=en#zippy=%2Chow-password-protection-works%2Chow-we-protect-your-data

If you want the technical details, please take a look at this paper.  It
goes into detail about the process for Chrome, Firefox, and LastPass.

https://courses.csail.mit.edu/6.857/2020/projects/6-Vadari-Maccow-Lin-Baral.pdf

Scott Helms



On Sat, Jun 12, 2021 at 5:51 PM William Herrin  wrote:

> On Sat, Jun 12, 2021 at 12:10 PM K. Scott Helms 
> wrote:
> >   Scott, Google's computer is able to compose an html document which
> > contains my passwords in plain text. Whatever dance they do to either
> > side of that point in their process, at that point they possess my
> > passwords in plain text. Why is this concept a mystery to anyone?
> >
> > Because it's wrong, they don't have your passwords you do (more
> accurately your device does).  They don't combine the decryption keys with
> the encrypted data, your device does.
>
> Look buddy, I'm not lying. Google's server at passwords.google.com
> composed an html web page containing my plaintext passwords and sent
> it to me. Not decrypted by my browser after combining it with a
> locally stored key. Decrypted on and by Google's server. It's not
> wrong. It's not false. It happened just like that.
>
>
> > You did authorize, you just didn't read the fine print.
>
> I always read the fine print. I'm that guy. I don't always go
> searching the menus for bad defaults but I always read everything they
> bother to tell me I'm agreeing to.
>
> Regards,
> Bill Herrin
>
>
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/
>

Re: Google uploading your plain text passwords

[K. Scott Helms]

  Scott, Google's computer is able to compose an html document which
contains my passwords in plain text. Whatever dance they do to either
side of that point in their process, at that point they possess my
passwords in plain text. Why is this concept a mystery to anyone?

Because it's wrong, they don't have your passwords you do (more accurately
your device does).  They don't combine the decryption keys with the
encrypted data, your device does. This is the case whenever something is
encrypted rather than hashed.  It's literally impossible to provide a
password saving mechanism that hashes the credentials.


  If I had authorized it, it would indeed be just like any other
password managing web site. I did not knowingly authorize it. They
snuck it on me.

You did authorize, you just didn't read the fine print.  Having said that,
this part of your complaint is definitely the one that has the most merit
IMO since if you enable it on mobile it directs you to a web page that you
can't see at that time.

If you're concerned then I'd recommend setting a synch phrase, which makes
it impossible for Google to decrypt the credentials without you inputting
it and they do not store it.

https://support.google.com/chrome/answer/165139?visit_id=637591216572649483-884903087=1

Scott Helms



On Sat, Jun 12, 2021 at 10:29 AM William Herrin  wrote:

> On Sat, Jun 12, 2021 at 5:11 AM K. Scott Helms 
> wrote:
> > Encryption != plain text, just because it's not a hash doesn't mean it's
> problematic (if done correctly).
>
> Scott, Google's computer is able to compose an html document which
> contains my passwords in plain text. Whatever dance they do to either
> side of that point in their process, at that point they possess my
> passwords in plain text. Why is this concept a mystery to anyone?
>
>
> > This is the exact same method that every single password management
> system uses and all are far better for the average user than trying to
> reuse a single password or write them down.
>
> If I had authorized it, it would indeed be just like any other
> password managing web site. I did not knowingly authorize it. They
> snuck it on me.
>
> Regards,
> Bill Herrin
>
>
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/
>

Re: Google uploading your plain text passwords

[K. Scott Helms]

Encryption != plain text, just because it's not a hash doesn't mean it's
problematic (if done correctly).  This is the exact same method that every
single password management system uses and all are far better for the
average user than trying to reuse a single password or write them down.

Scott Helms



On Fri, Jun 11, 2021 at 12:53 PM William Herrin  wrote:

> On Fri, Jun 11, 2021 at 9:42 AM César de Tassis Filho
>  wrote:
> > Google does not have access to your plain-text passwords in either case.
>
> If they can display the plain text passwords to me on my screen in a
> non-Google web browser then they have access to my plain text
> passwords. Everything else is semantics.
>
> Regards,
> Bill Herrin
>
>
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/
>

Re: Parler

[K. Scott Helms]

They certainly have been many many times, but that's an entirely
different animal than the rules for content hosting and publishing.
Actions from network providers have (AFAIK) always been in conjunction
with some traffic from or to the violating party rather than an
otherwise legal content hosting arrangement.


Scott Helms


On Sun, Jan 10, 2021 at 9:05 PM mark seery  wrote:
>
> I assume multiple networks/ ISPs that have acceptable use policies that call 
> out criminality and incitement to violence, for example:
>
> https://www.xfinity.com/support/articles/comcast-acceptable-use-policy
>
> Have these AUPs been invoked previously for these reasons, or would that be 
> new territory?
>
> Sent from Mobile Device
>
> On Jan 10, 2021, at 2:52 PM, K. Scott Helms  wrote:
>
> 
> Right, it's not a list for content hosting.
>
> Scott Helms
>
> On Sun, Jan 10, 2021, 5:42 PM  wrote:
>>
>> No, this is a list for Network Operators.
>>
>> Sent from my iPhone
>>
>> On Jan 10, 2021, at 5:32 PM, K. Scott Helms  wrote:
>>
>> 
>> This is a list for pushing bits.  The fact that many/most of us have other 
>> businesses doesn't make this an appropriate forum for SIP issues (to use my 
>> own work as an example).
>>
>> On Sun, Jan 10, 2021, 4:52 PM  wrote:
>>>
>>> This is a list for Network Operators, AWS certainly operates networks.
>>>
>>> Sent from my iPhone
>>>
>>> On Jan 10, 2021, at 4:27 PM, K. Scott Helms  wrote:
>>>
>>> 
>>> No,
>>>
>>> It really does not.  Section 230 only applies to publishers, and not to 
>>> network providers.  If this were a cloud hosting provider list then you'd 
>>> be correct, but as a network provider's list it does not belong here.
>>>
>>>
>>> Scott Helms
>>>
>>>
>>>
>>> On Sun, Jan 10, 2021 at 3:21 PM Lady Benjamin PD Cannon  
>>> wrote:
>>>>
>>>> As network operations and compute/cloud/hosting operations continue to 
>>>> coalesce, I very much disagree with you.  Section 230 is absolutely 
>>>> relevant, this discussion is timely and relevant, and it directly affects 
>>>> me as both a telecom and cloud compute/services provider.
>>>>
>>>>
>>>> —L.B.
>>>>
>>>> Lady Benjamin PD Cannon, ASCE
>>>> 6x7 Networks & 6x7 Telecom, LLC
>>>> CEO
>>>> b...@6by7.net
>>>> "The only fully end-to-end encrypted global telecommunications company in 
>>>> the world.”
>>>> FCC License KJ6FJJ
>>>>
>>>> 
>>>> 
>>>>
>>>> On Jan 10, 2021, at 12:13 PM, K. Scott Helms  
>>>> wrote:
>>>>
>>>> It's not, and frankly it's disappointing to see people pushing an agenda 
>>>> here.
>>>>
>>>>
>>>> Scott Helms
>>>>
>>>>
>>>> On Sun, Jan 10, 2021 at 9:37 AM  wrote:
>>>>
>>>>
>>>> NANOG is a group of Operators, discussion does not have to be about 
>>>> networking. I have already explained how this represents a significant 
>>>> issue for Network Operators.
>>>>
>>>> On Jan 10, 2021, at 9:09 AM, Mike Bolitho  wrote:
>>>>
>>>> 
>>>> It has nothing to do with networking. Their decision was necessarily 
>>>> political. If you can specifically bring up an issue, beyond speculative, 
>>>> on how their new chosen CDN is somehow now causing congestion or routing 
>>>> issues on the public internet, then great. But as of now, that isn't even 
>>>> a thing. It's just best to leave it alone because it will devolve into 
>>>> chaos.
>>>>
>>>> - Mike Bolitho
>>>>
>>>> On Sun, Jan 10, 2021, 6:54 AM  wrote:
>>>>
>>>>
>>>> Why? This is extremely relevant to network operators and is not political 
>>>> at all.
>>>>
>>>> On Jan 10, 2021, at 8:51 AM, Mike Bolitho  wrote:
>>>>
>>>> 
>>>> Can we please not go down this rabbit hole on here? List admins?
>>>>
>>>> - Mike Bolitho
>>>>
>>>> On Sun, Jan 10, 2021, 1:26 AM William Herrin  wrote:
>>>>
>>>>
>>>> Anybody looking for a new customer opportunity? It seems Parler is in
>>>> search of a new service provider. Vendors need only provide all the
>>>> proprietary AWS APIs that Parler depends upon to function.
>>>>
>>>> https://www.washingtonpost.com/technology/2021/01/09/amazon-parler-suspension/
>>>>
>>>> Regards,
>>>> Bill HErrin
>>>>
>>>>

Re: Parler

[K. Scott Helms]

Right, it's not a list for content hosting.

Scott Helms

On Sun, Jan 10, 2021, 5:42 PM  wrote:

> No, this is a list for Network Operators.
>
> Sent from my iPhone
>
> On Jan 10, 2021, at 5:32 PM, K. Scott Helms 
> wrote:
>
> 
> This is a list for pushing bits.  The fact that many/most of us have other
> businesses doesn't make this an appropriate forum for SIP issues (to use my
> own work as an example).
>
> On Sun, Jan 10, 2021, 4:52 PM  wrote:
>
>> This is a list for Network Operators, AWS certainly operates networks.
>>
>> Sent from my iPhone
>>
>> On Jan 10, 2021, at 4:27 PM, K. Scott Helms 
>> wrote:
>>
>> 
>> No,
>>
>> It really does not.  Section 230 only applies to publishers, and not to
>> network providers.  If this were a cloud hosting provider list then you'd
>> be correct, but as a network provider's list it does not belong here.
>>
>>
>> Scott Helms
>>
>>
>>
>> On Sun, Jan 10, 2021 at 3:21 PM Lady Benjamin PD Cannon 
>> wrote:
>>
>>> As network operations and compute/cloud/hosting operations continue to
>>> coalesce, I very much disagree with you.  Section 230 is absolutely
>>> relevant, this discussion is timely and relevant, and it directly affects
>>> me as both a telecom and cloud compute/services provider.
>>>
>>>
>>> —L.B.
>>>
>>> Lady Benjamin PD Cannon, ASCE
>>> 6x7 Networks & 6x7 Telecom, LLC
>>> CEO
>>> b...@6by7.net
>>> "The only fully end-to-end encrypted global telecommunications company
>>> in the world.”
>>> FCC License KJ6FJJ
>>>
>>> 
>>> 
>>>
>>> On Jan 10, 2021, at 12:13 PM, K. Scott Helms 
>>> wrote:
>>>
>>> It's not, and frankly it's disappointing to see people pushing an agenda
>>> here.
>>>
>>>
>>> Scott Helms
>>>
>>>
>>> On Sun, Jan 10, 2021 at 9:37 AM  wrote:
>>>
>>>
>>> NANOG is a group of Operators, discussion does not have to be about
>>> networking. I have already explained how this represents a significant
>>> issue for Network Operators.
>>>
>>> On Jan 10, 2021, at 9:09 AM, Mike Bolitho  wrote:
>>>
>>> 
>>> It has nothing to do with networking. Their decision was necessarily
>>> political. If you can specifically bring up an issue, beyond speculative,
>>> on how their new chosen CDN is somehow now causing congestion or routing
>>> issues on the public internet, then great. But as of now, that isn't even a
>>> thing. It's just best to leave it alone because it will devolve into chaos.
>>>
>>> - Mike Bolitho
>>>
>>> On Sun, Jan 10, 2021, 6:54 AM  wrote:
>>>
>>>
>>> Why? This is extremely relevant to network operators and is not
>>> political at all.
>>>
>>> On Jan 10, 2021, at 8:51 AM, Mike Bolitho  wrote:
>>>
>>> 
>>> Can we please not go down this rabbit hole on here? List admins?
>>>
>>> - Mike Bolitho
>>>
>>> On Sun, Jan 10, 2021, 1:26 AM William Herrin  wrote:
>>>
>>>
>>> Anybody looking for a new customer opportunity? It seems Parler is in
>>> search of a new service provider. Vendors need only provide all the
>>> proprietary AWS APIs that Parler depends upon to function.
>>>
>>>
>>> https://www.washingtonpost.com/technology/2021/01/09/amazon-parler-suspension/
>>>
>>> Regards,
>>> Bill HErrin
>>>
>>>
>>>

Re: Parler

[K. Scott Helms]

This is a list for pushing bits.  The fact that many/most of us have other
businesses doesn't make this an appropriate forum for SIP issues (to use my
own work as an example).

On Sun, Jan 10, 2021, 4:52 PM  wrote:

> This is a list for Network Operators, AWS certainly operates networks.
>
> Sent from my iPhone
>
> On Jan 10, 2021, at 4:27 PM, K. Scott Helms 
> wrote:
>
> 
> No,
>
> It really does not.  Section 230 only applies to publishers, and not to
> network providers.  If this were a cloud hosting provider list then you'd
> be correct, but as a network provider's list it does not belong here.
>
>
> Scott Helms
>
>
>
> On Sun, Jan 10, 2021 at 3:21 PM Lady Benjamin PD Cannon 
> wrote:
>
>> As network operations and compute/cloud/hosting operations continue to
>> coalesce, I very much disagree with you.  Section 230 is absolutely
>> relevant, this discussion is timely and relevant, and it directly affects
>> me as both a telecom and cloud compute/services provider.
>>
>>
>> —L.B.
>>
>> Lady Benjamin PD Cannon, ASCE
>> 6x7 Networks & 6x7 Telecom, LLC
>> CEO
>> b...@6by7.net
>> "The only fully end-to-end encrypted global telecommunications company in
>> the world.”
>> FCC License KJ6FJJ
>>
>> 
>> 
>>
>> On Jan 10, 2021, at 12:13 PM, K. Scott Helms 
>> wrote:
>>
>> It's not, and frankly it's disappointing to see people pushing an agenda
>> here.
>>
>>
>> Scott Helms
>>
>>
>> On Sun, Jan 10, 2021 at 9:37 AM  wrote:
>>
>>
>> NANOG is a group of Operators, discussion does not have to be about
>> networking. I have already explained how this represents a significant
>> issue for Network Operators.
>>
>> On Jan 10, 2021, at 9:09 AM, Mike Bolitho  wrote:
>>
>> 
>> It has nothing to do with networking. Their decision was necessarily
>> political. If you can specifically bring up an issue, beyond speculative,
>> on how their new chosen CDN is somehow now causing congestion or routing
>> issues on the public internet, then great. But as of now, that isn't even a
>> thing. It's just best to leave it alone because it will devolve into chaos.
>>
>> - Mike Bolitho
>>
>> On Sun, Jan 10, 2021, 6:54 AM  wrote:
>>
>>
>> Why? This is extremely relevant to network operators and is not political
>> at all.
>>
>> On Jan 10, 2021, at 8:51 AM, Mike Bolitho  wrote:
>>
>> 
>> Can we please not go down this rabbit hole on here? List admins?
>>
>> - Mike Bolitho
>>
>> On Sun, Jan 10, 2021, 1:26 AM William Herrin  wrote:
>>
>>
>> Anybody looking for a new customer opportunity? It seems Parler is in
>> search of a new service provider. Vendors need only provide all the
>> proprietary AWS APIs that Parler depends upon to function.
>>
>>
>> https://www.washingtonpost.com/technology/2021/01/09/amazon-parler-suspension/
>>
>> Regards,
>> Bill HErrin
>>
>>
>>

Re: Parler

[K. Scott Helms]

It's not, and frankly it's disappointing to see people pushing an agenda here.


Scott Helms


On Sun, Jan 10, 2021 at 9:37 AM  wrote:
>
> NANOG is a group of Operators, discussion does not have to be about 
> networking. I have already explained how this represents a significant issue 
> for Network Operators.
>
> On Jan 10, 2021, at 9:09 AM, Mike Bolitho  wrote:
>
> 
> It has nothing to do with networking. Their decision was necessarily 
> political. If you can specifically bring up an issue, beyond speculative, on 
> how their new chosen CDN is somehow now causing congestion or routing issues 
> on the public internet, then great. But as of now, that isn't even a thing. 
> It's just best to leave it alone because it will devolve into chaos.
>
> - Mike Bolitho
>
> On Sun, Jan 10, 2021, 6:54 AM  wrote:
>>
>> Why? This is extremely relevant to network operators and is not political at 
>> all.
>>
>> On Jan 10, 2021, at 8:51 AM, Mike Bolitho  wrote:
>>
>> 
>> Can we please not go down this rabbit hole on here? List admins?
>>
>> - Mike Bolitho
>>
>> On Sun, Jan 10, 2021, 1:26 AM William Herrin  wrote:
>>>
>>> Anybody looking for a new customer opportunity? It seems Parler is in
>>> search of a new service provider. Vendors need only provide all the
>>> proprietary AWS APIs that Parler depends upon to function.
>>>
>>> https://www.washingtonpost.com/technology/2021/01/09/amazon-parler-suspension/
>>>
>>> Regards,
>>> Bill HErrin

Re: Centurylink having a bad morning?

[K. Scott Helms]

We've been on hold for more than an hour trying to get an update.

We see the same behavior where they continue to announce our blocks
despite all the interfaces to them being hard down.

Scott Helms


On Sun, Aug 30, 2020 at 8:58 AM Jason Kuehl  wrote:
>
> Well, When I tried calling I got a fast busy, so that's nice.
>
> On Sun, Aug 30, 2020 at 8:33 AM David Hubbard  
> wrote:
>>
>> Same.  Also, as reported on outages list, what’s even worse is that they 
>> appear to be continuing to propagate advertisements from circuits whose 
>> sessions have been turned down.  I validated ours still were via a couple 
>> looking glass portals.  Down Detector shows nearly every major service 
>> provider impacted.
>>
>>
>>
>> They’re not reachable so who knows if they’re even working on it.  I feel 
>> like they’ve been cutting heavily on the network ops side in recent years…
>>
>>
>>
>> From: NANOG  on 
>> behalf of Drew Weaver via NANOG 
>> Reply-To: Drew Weaver 
>> Date: Sunday, August 30, 2020 at 8:23 AM
>> To: "nanog@nanog.org" 
>> Subject: Centurylink having a bad morning?
>>
>>
>>
>> Hello,
>>
>>
>>
>> Woke up this morning to a bunch of reports of issues with connectivity had 
>> to shut down some Level3/CTL connections to get it to return to normal.
>>
>>
>>
>> As of right now their support portal won’t load: 
>> https://www.centurylink.com/business/login/
>>
>>
>>
>> Just wondering what others are seeing.
>>
>>
>
>
>
> --
> Sincerely,
>
> Jason W Kuehl
> Cell 920-419-8983
> jason.w.ku...@gmail.com

Re: TCP and UDP Port 0 - Should an ISP or ITP Block it?

[K. Scott Helms]

Nick,

Data on blocking inbound TCP or the kinds of gear that mistakenly
labels UDP fragments as DST port 0?

Scott Helms


On Wed, Aug 26, 2020 at 9:00 AM Nick Hilliard  wrote:
>
> K. Scott Helms wrote on 26/08/2020 13:55:
> > To be clear, UDP port 0 is not and probably shouldn't be blocked
> > because some network gear and reporting tools may mistake a fragmented
> > UDP PDU for port 0.  That's an implementation error, but one that may
> > be common enough to create issues for users.
> do you have data on this?
>
> Nick
>

Re: TCP and UDP Port 0 - Should an ISP or ITP Block it?

[K. Scott Helms]

To be clear, UDP port 0 is not and probably shouldn't be blocked
because some network gear and reporting tools may mistake a fragmented
UDP PDU for port 0.  That's an implementation error, but one that may
be common enough to create issues for users.  Blocking inbound TCP
port 0 is something that I've personally done in dozens of ISP
networks over more than a decade without a single reported issue.

Scott Helms


On Tue, Aug 25, 2020 at 7:39 PM narhiro  wrote:
>
>
> > "Port 0 is a reserved port, which means it should not be used by
> > applications. Network abuse has prompted the need to block this port."
> >
> > "What about UDP IP fragmentation?"
> >
> > I'm not sure I follow this.  The IP packet will be fragmented with UDP
> > inside it.  When the IP packet gets put together the UDP PDU will have
> > a port number.  It's possible that some packet analyzers or network
> > gear will improperly "see" a partial UDP flow as port 0 but that's a
> > mischaracterization of the flow.
> >
> >
> > Scott Helms
> >
> > Scott Helms
> >
> >
> >
> >>> On Tue, Aug 25, 2020 at 8:17 AM Job Snijders  wrote:
> >>>
> >>>> On Tue, Aug 25, 2020 at 07:27:33AM -0400, K. Scott Helms wrote:
> >>> I think a fairly easy thing to do is see what other large retail ISPs
> >>> have done.  Comcast, as an example, lists all of the ports they block
> >>> and 0 is blocked.  I do recommend that port 0 be blocked by all of the
> >>> ISPs I work with and frankly Comcast's list is a pretty good one to
> >>> use in general, though you will get some pushback on things like SMTP.
> >>> https://www.xfinity.com/support/articles/list-of-blocked-ports
> >>
> >> I may be reading the table incorrectly, but it seems to me Comcast is
> >> *not* blocking UDP port 0 according to the above URL?
> >>
> >>> Transit providers are a little bit different, but then again port 0 is
> >>> also different since AFAIK it's never had a legitimate use case.  It's
> >>> always been a reserved port.  I'd personally block it if I ran a
> >>> transit, but I'd be more willing to open it up for one of my large
> >>> customers (in a limited way) than I would on the retail side.
> >>> https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml
> >>
> >> What about UDP IP fragmentation?
> >>
> >> Kind regards,
> >>
> >> Job

Re: TCP and UDP Port 0 - Should an ISP or ITP Block it?

[K. Scott Helms]

That's correct, I can only blame my lack of coffee at that point for
the oversight.  I went back and looked at where we have this
implemented and it's only TCP.


Scott Helms


On Tue, Aug 25, 2020 at 8:46 AM Job Snijders  wrote:
>
> On Tue, Aug 25, 2020 at 08:27:24AM -0400, K. Scott Helms wrote:
> > Comcast is blocking it.  From the table on that page.
> >
> > "Port 0 is a reserved port, which means it should not be used by
> > applications. Network abuse has prompted the need to block this port."
>
> The 'Transport' column seems to indicate that TCP port 0 is blocked, but
> not that UDP port 0 is blocked. I believe there are comcast people on
> this mailing list, it would be interesting to hear what the
> considerations were to block one but not the other.
>
> > "What about UDP IP fragmentation?"
> >
> > I'm not sure I follow this.  The IP packet will be fragmented with UDP
> > inside it.  When the IP packet gets put together the UDP PDU will have
> > a port number.  It's possible that some packet analyzers or network
> > gear will improperly "see" a partial UDP flow as port 0 but that's a
> > mischaracterization of the flow.
>
> You are absolutely right. There is no layer-4 header in a fragment.
> 'port 0' in netflow/ipfix traffic analyzer tools when displayed may be
> the result of a lack of ability to label it differently in the
> datastructures used. "mischaracterization" is a fitting word :-)
>
> Kind regards,
>
> Job

Re: TCP and UDP Port 0 - Should an ISP or ITP Block it?

[K. Scott Helms]

Job,

Comcast is blocking it.  From the table on that page.

"Port 0 is a reserved port, which means it should not be used by
applications. Network abuse has prompted the need to block this port."

"What about UDP IP fragmentation?"

I'm not sure I follow this.  The IP packet will be fragmented with UDP
inside it.  When the IP packet gets put together the UDP PDU will have
a port number.  It's possible that some packet analyzers or network
gear will improperly "see" a partial UDP flow as port 0 but that's a
mischaracterization of the flow.


Scott Helms

Scott Helms



On Tue, Aug 25, 2020 at 8:17 AM Job Snijders  wrote:
>
> On Tue, Aug 25, 2020 at 07:27:33AM -0400, K. Scott Helms wrote:
> > I think a fairly easy thing to do is see what other large retail ISPs
> > have done.  Comcast, as an example, lists all of the ports they block
> > and 0 is blocked.  I do recommend that port 0 be blocked by all of the
> > ISPs I work with and frankly Comcast's list is a pretty good one to
> > use in general, though you will get some pushback on things like SMTP.
> >
> > https://www.xfinity.com/support/articles/list-of-blocked-ports
>
> I may be reading the table incorrectly, but it seems to me Comcast is
> *not* blocking UDP port 0 according to the above URL?
>
> > Transit providers are a little bit different, but then again port 0 is
> > also different since AFAIK it's never had a legitimate use case.  It's
> > always been a reserved port.  I'd personally block it if I ran a
> > transit, but I'd be more willing to open it up for one of my large
> > customers (in a limited way) than I would on the retail side.
> >
> > https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml
>
> What about UDP IP fragmentation?
>
> Kind regards,
>
> Job

Re: TCP and UDP Port 0 - Should an ISP or ITP Block it?

[K. Scott Helms]

Douglas,

I think a fairly easy thing to do is see what other large retail ISPs have
done.  Comcast, as an example, lists all of the ports they block and 0 is
blocked.  I do recommend that port 0 be blocked by all of the ISPs I work
with and frankly Comcast's list is a pretty good one to use in general,
though you will get some pushback on things like SMTP.

https://www.xfinity.com/support/articles/list-of-blocked-ports

Transit providers are a little bit different, but then again port 0 is also
different since AFAIK it's never had a legitimate use case.  It's always
been a reserved port.  I'd personally block it if I ran a transit, but I'd
be more willing to open it up for one of my large customers (in a limited
way) than I would on the retail side.

https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml


Scott Helms



On Tue, Aug 25, 2020 at 7:16 AM Douglas Fischer 
wrote:

> I think that the subject of the e-mail is very self-explanatory.
>
> With some analysis of what is running over our network, ISP or ITP, we
> will be able to see some TCP/UDP(mostly UDP) packets with source or
> destination to port 0.
>
> I can think of a genuine use of it.
> (Maybe someone cloud help me see what I'm not seen.)
>
> So I have two questions:
>
> a) Should an ISP block that Kind of traffic?
> (like anti-spoofing on BNG/B-RAS)
>
> b) Should a Transit Provider block that Kind of traffic?
>
>
> --
> Douglas Fernando Fischer
> Engº de Controle e Automação
>

Re: How to manage Static IPs to customers

[K. Scott Helms]

The spec allows for bridging or layer 3 but none of the major or certified
manufacturers support bridging on larger platforms.  (>1000 modems)

Scott Helms



On Fri, May 8, 2020 at 3:56 PM Brandon Martin 
wrote:

> I'm curious...
>
> Is it part of the DOCSIS spec that the CMTS terminates L3, or can they
> bridge to IEEE 802(.3) and delegate that to some other piece of gear?
> I'm unfortunately not familiar with the MSO world much at all aside from
> a little bit of L1.
>
> --
> Brandon Martin
>

Re: How to manage Static IPs to customers

[K. Scott Helms]

Javier,

There's really no good way to handle this without routing or tunneling that
I've been able to find in a very long time.  (SD-WAN can help, but it's
just a fancy way to tunnel in this regard.)  It's pretty amazing that this
is such an issue, but it remains so.  I have tried to work around this
using BSoD (
https://specification-search.cablelabs.com/business-services-over-docsis-layer-2-virtual-private-networks
)
but we eventually abandoned the effort because it rapidly became to
expensive to scale to solve a niche problem.

Scott Helms



On Fri, May 8, 2020 at 8:58 AM Javier Gutierrez Guerra <
guer...@westmancom.com> wrote:

> That's surprising to me, I have no intentions to do routing with our cable
> subscribers, that seems like a headache for both sides
> Today we have specific ranges within subnets from where we assign IPs to
> customers, my main problem that I'm trying to get around is having to
> change a customer static IP if their node gets splitter and I have to mode
> them to a different CMTS
>
> Thanks,
>
> Javier Gutierrez Guerra
>
>
>
> -Original Message-
> From: NANOG  On Behalf Of Bryan Fields
> Sent: Thursday, May 7, 2020 5:57 PM
> To: nanog@nanog.org
> Subject: Re: How to manage Static IPs to customers
>
> CAUTION: This email is from an external source. Do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
> On 5/7/20 5:54 PM, Brandon Jackson via NANOG wrote:
> > I have seen (Charter) and heard quite a few run RIP or some other
> > routing protocol on the CPE.
>
> Yep, it's RIP.  They don't support IPv6 on this either.  I've been asking
> for
> IPv6 since 2006, it's always next year.
>
> --
> Bryan Fields
>
> 727-409-1194 - Voice
> http://bryanfields.net
>

Re: Backup over 4G/LTE

[K. Scott Helms]

There are lots of options to solve that problem.

Peplink, 128T, Viptela (Cisco), Velocloud (VMWare), etc.

Scott Helms



On Tue, Jan 28, 2020 at 6:31 PM K MEKKAOUI  wrote:

> Dear NANOG Community,
>
>
>
> Can anyone help with any device information that provides redundancy for
> business internet access? In other words when the internet provided through
> the cable modem fails the 4G/LTE takes over automatically to provide
> internet access to the client.
>
>
>
> Thank you
>
>
>
> KARIM M.
>
>
>

Re: This DNS over HTTP thing

[K. Scott Helms]

They almost have to change the default since there are (comparatively) very
few DoH providers compared to DNS providers.

On Tue, Oct 1, 2019, 2:40 PM Damian Menscher via NANOG 
wrote:

> On Tue, Oct 1, 2019 at 12:24 PM Jay R. Ashworth  wrote:
>
>> - Original Message -
>> > From: "Stephane Bortzmeyer" 
>> > To: "Jeroen Massar" 
>>
>> >> While the 'connection to the recursor' is 'encrypted', the recursor
>> >> is still in clear text... one just moves who can see what you are
>> >> doing with this.
>> >
>> > As with any cryptographic protocol. Same thing with VPNs, SSH and
>> > whatever: the remote end can see what you do. What's your point?
>>
>> I'm still assimilating this, but based on what I've read this half hour,
>> his point is that "*it's none of Alphabet's damn business* where I go that
>> isn't Google".
>>
>
> What's missing from this discussion are some basic facts, like "is Google
> going to change your DNS settings to 8.8.8.8?"
>
> The opening paragraph of
> https://blog.chromium.org/2019/09/experimenting-with-same-provider-dns.html
>  reads:
>
> "This experiment will be done in collaboration with DNS providers who
> already support DoH, with the goal of improving our mutual users’ security
> and privacy by upgrading them to the DoH version of their current DNS
> service. With our approach, the DNS service used will not change, only the
> protocol will. As a result, existing content controls of your current DNS
> provider, including any existing protections for children, will remain
> active."
>
> Could someone provide a reference of Google saying they'll change the
> default nameserver?  Without that, I think all of Jeroen's arguments fall
> apart?
>
> Damian
>

Re: Packetstream - how does this not violate just about every provider's ToS?

[K. Scott Helms]

After all, it worked for Napster


Scott Helms



On Thu, Apr 25, 2019 at 3:23 PM John Levine  wrote:

> In article  you write:
> >-=-=-=-=-=-
> >
> >feeling cranky, are we, job?   (accusing an antispam expert of spamming
> on a mailing list by having too long a .sig?)
> >but it’s true!  anne runs the internet, and the rest of us (except for
> ICANN GAC representatives) all accept that.
> >
> >to actually try to make a more substantial point, i am quite curious how
> the AUPs of carriers try to disallow
> >bandwidth resale while permitting
> >
> >• cybercafe operations and other “free wifi" (where internet service
> might be provided for patrons in a
> >hotel or cafe)
> >• wireless access point schemes where you make money or get credit for
> allowing use of your bandwidth (e.g. Fon)
> >• other proxy services that use bandwidth such as tor exit nodes and
> openvpn gateways
>
> To belabor the fairly obvious, residential and business service are
> different even if the technology is the same.  For example, Comcast's
> residential TOS says:
>
>   You agree that the Service(s) and the Xfinity Equipment will be used
>   only for personal, residential, non-commercial purposes, unless
>   otherwise specifically authorized by us in writing. You are prohibited
>   from reselling or permitting another to resell the Service(s) in whole
>   or in part, ... [ long list of other forbidden things ]
>
> Their business TOS is different.  It says no third party use unless
> your agreement permits it, so I presume they have a coffee shop plan.
> (The agreements don't seem to be on their web site.)  I'd also observe
> that coffee shop wifi isn't "resale" since it's free, it's an amenity.
>
> As to how do these guys think they'll get away with it, my guess is
> that they heard that "disruption" means ignoring laws and contracts
> and someone told them that is a good thing.
>
> R's,
> John
>

Re: Comcast storing WiFi passwords in cleartext?

[K. Scott Helms]

Tom,

No, and I would hope that they were storing it in an encrypted format and
then decrypting it on the fly for display in the customer portal.

Scott Helms



On Thu, Apr 25, 2019 at 1:55 PM Tom Beecher  wrote:

> As much as it pains me to Devil's Advocate for Comcast... Has anyone
> proven that they are storing this PSK in cleartext? From the original
> StackExchange post :
>
> " When I went to the account web page, it showed me my password. I
> changed the password and it instantly showed the new password on the
> account web page (after refresh). "
>
> The SNMP response is essentially cleartext , sure. But perhaps they are
> performing the query from a modem management network only accessible from
> the RF side, the transmission back to the CS backend is encrypted in
> flight, and the data is also encrypted at rest until retrieved and
> decrypted by a agent or the end user via the web portal. Nothing has been
> shown that I can recall reading that proves or disproves any of that.
>
>
>
> On Thu, Apr 25, 2019 at 1:17 PM Doug Barton  wrote:
>
>> On 4/25/19 8:04 AM, K. Scott Helms wrote:
>> > Just so you know, if you have an embedded router from a service
>> provider
>> > all of that data is _already_ being transmitted and has been for a long
>> > long time.
>>
>> Responding to a pseudo-random message ...
>>
>> If you are an average consumer and purchase a managed solution (in this
>> case a WAP that comes as part of your package) I think it's perfectly
>> reasonable for the vendor to manage it accordingly, even if said
>> consumer doesn't fully understand the implications of that decision.
>>
>> In my mind, the problem here is not that the vendor has access to this
>> data, it's that they are STORING it in the first place, and storing it
>> in the clear to boot. In the hypothetical service call that we've
>> speculated is the driver for this, the extra 15 or 20 seconds that it
>> would take to pull the data via SNMP is in the noise.
>>
>> There are two mindsets that desperately need changing in the tech world:
>>
>> 1. Do not store data that you don't have a legitimate requirement to store
>> 2. Do not store anything even remotely sensitive in the clear
>>
>> We live in a world of all breaches, all of the time. So we need to start
>> thinking not in terms of just protecting said data from the outside, but
>> rather in terms of limiting the attack surface to start with, and
>> protecting the data at rest. So that WHEN there is a breach, whether
>> from within or without, the damage will be minimal.
>>
>> As many have pointed out, this information is freely available via SNMP,
>> so it's a classic example of something that didn't need to be stored in
>> the first place.
>>
>> Doug
>>
>

Re: Comcast storing WiFi passwords in cleartext?

[K. Scott Helms]

Doug,

I don't disagree, but things are pretty complicated, much more so than they
might seem from the outside.  First, if the configuration isn't stored
there's literally no way to have a backup for most of the CPE vendors so
there's definitely reason to have it duplicated in the service providers'
systems.  Very few allow for end users to download their router
configuration via the admin page and I know of none that encrypt that
configuration before it is delivered to the end user's computer.  (It's
also relevant that the usage for those vendors that do allow end users to
backup the config is vanishingly low.)  If we're looking at a TR-069 based
system for managing the WiFi and router components it's not really feasible
to do a real time grab of that data since that process can take up to ~5
minutes depending on your periodic inform settings in your ACS.  That's
because TR-069 is inherently a push technology (from the CPE to the ACS)
rather than a pull like SNMP.

The next piece is that a DOCSIS configuration file itself, which in some
cases contains these parameters, is by the standard required to be
delivered via insecure protocols, namely TFTP.  Newer devices have options
to allow for TLS secured HTTP, but that's very rare today in production
provisioning systems, and in case the secured protocols are all still
optional in the spec.  In general the config file itself is stored in it's
text format on the provisioning systems or if the file is dynamically
generated the user specific parameters are held in a database with the
general ones coming from a template for that class of service.

Again, I'm not disagreeing with your premise but the service providers
directly control a small piece of the overall process and we're still
working with standards from earlier times.  Most cable operators have
gotten rid of their DOCSIS 2.0 (1.0 and 1.1 as well) but it's not uncommon
to find a handful of users with (mostly customer owned) D2 devices that the
provisioning and OSS systems still have to deal with.  DOCSIS 3.0 devices
are the majority and 3.1 devices are just now being rolled out in large
numbers.

In short, not everything is quickly retrievable, much of the configuration
is in fact generated by the service provider and must be maintained because
the modem needs to download its configuration every time it reboots, and
the vendors and associations in the provisioning and OSS space have more
input than the operators themselves.

Scott Helms



On Thu, Apr 25, 2019 at 1:16 PM Doug Barton  wrote:

> On 4/25/19 8:04 AM, K. Scott Helms wrote:
> > Just so you know, if you have an embedded router from a service provider
> > all of that data is _already_ being transmitted and has been for a long
> > long time.
>
> Responding to a pseudo-random message ...
>
> If you are an average consumer and purchase a managed solution (in this
> case a WAP that comes as part of your package) I think it's perfectly
> reasonable for the vendor to manage it accordingly, even if said
> consumer doesn't fully understand the implications of that decision.
>
> In my mind, the problem here is not that the vendor has access to this
> data, it's that they are STORING it in the first place, and storing it
> in the clear to boot. In the hypothetical service call that we've
> speculated is the driver for this, the extra 15 or 20 seconds that it
> would take to pull the data via SNMP is in the noise.
>
> There are two mindsets that desperately need changing in the tech world:
>
> 1. Do not store data that you don't have a legitimate requirement to store
> 2. Do not store anything even remotely sensitive in the clear
>
> We live in a world of all breaches, all of the time. So we need to start
> thinking not in terms of just protecting said data from the outside, but
> rather in terms of limiting the attack surface to start with, and
> protecting the data at rest. So that WHEN there is a breach, whether
> from within or without, the damage will be minimal.
>
> As many have pointed out, this information is freely available via SNMP,
> so it's a classic example of something that didn't need to be stored in
> the first place.
>
> Doug
>

Re: Comcast storing WiFi passwords in cleartext?

[K. Scott Helms]

Just so you know, if you have an embedded router from a service provider
all of that data is _already_ being transmitted and has been for a long
long time.  If it's being collected via SNMPv2c it is being transmitted in
the clear (though hopefully encrypted via BPI+ between the modem and the
CMTS).  If it's being collected via TR-069 it _may_ (should be) encrypted
in transit but in my experience that isn't guaranteed and when its being
sent over TLS there's often a self signed cert in the chain.

Scott Helms



On Thu, Apr 25, 2019 at 10:45 AM Benjamin Sisco 
wrote:

> On 4/24/ 2019 10:34 AM, Seth Mattinen wrote:
>
> > That's looking at it from a technical perspective when it isn't a
> technical problem. People that buy "includes wifi" from their ISP often
> need extreme amounts of help with it, and thus the wifi credentials are
> stored and transmitted in plain text for tech support reasons.
>
> While I agree that the underlying need is to provide fast and effective
> customer service - it is ultimately a technical problem.  As it's been
> pointed out in subsequent posts WiFi is the leading cause of customer calls
> to an ISP offering the service.  Security and "ease of use" are often at
> odds with each other, and implementing the former with the latter is the
> challenge many of us wake up to each and every day.  The information should
> be encrypted at rest and in transit and could easily be decrypted by the
> CSP platform for use by customer support staff at the time of need when
> cusetomers call in - which would address the concern.
>
> In my experience, bad practice is easily replicated.  What else is
> transmitted in cleartext?  Today it's the WiFi password, tomorrow it's your
> login, port forwarding, DMZ, and other details that are far more useful to
> a remote attacker than your WiFi password.
>
>
>
>
> -Original Message-
> From: NANOG  On Behalf Of Seth Mattinen
> Sent: Wednesday, April 24, 2019 10:34 AM
> To: nanog@nanog.org
> Subject: Re: Comcast storing WiFi passwords in cleartext?
>
> Notice: This message originated outside of Just Associates. Verify the
> source & exercise caution with links and attachments.
>
> On 4/24/19 8:13 AM, Benjamin Sisco wrote:
> > The bigger concern should be the cleartext portion of the subject.
> There’s ZERO reason to store or transmit any credentials (login, service,
> keys, etc.), in any location, in an unencrypted fashion regardless of their
> perceived value or purpose.  Unless you like risk.
>
>
> That's looking at it from a technical perspective when it isn't a
> technical problem. People that buy "includes wifi" from their ISP often
> need extreme amounts of help with it, and thus the wifi credentials are
> stored and transmitted in plain text for tech support reasons.
>
> ~Seth
> Confidentiality Notice: This e-mail communication and any attachments may
> contain confidential and privi­leged information for the use of the
> designated recipients named above. If you are not the intended recipient,
> you are hereby notified that you have received this communication in error
> and that any review, disclosure, dissemination, distribution or copying of
> it or its contents is prohibited. If you have received this communica­tion
> in error, please notify me immediately by replying to this message and
> deleting it from your computer. Thank you.
>

Re: Comcast storing WiFi passwords in cleartext?

[K. Scott Helms]

James,

By the DOCSIS standard and every North American MSO's ToS I've seen (I've
worked with or for about 200 different cable operators over the last 20
years) your cable modem is always managed and the cable operator _always_
has access to its configuration and settings via SNMP.  The configuration
file for a DOCSIS cable modem is nothing more than a list of SNMP OIDs with
their values set the way the operator wants them.  This has been true since
DOCSIS 1.0 (which doesn't make it correct, just common).  Now, DOCSIS is
primarily deployed with SNMP version 2c though more and more operators,
especially the larger ones, are moving or have to SNMPv3.  I mention this
because every cable modem on a given CMTS that's deployed in SNMPv2c mode
will (should for proper functioning) have the same SNMP READ and WRITE
strings.  SNMPv2c traffic is clear text UDP with no standardized methods of
encryption available to the industry.  To mitigate this to an extent part
of the cable modem's configuration will (best practice anyway) have
filtering rules to only accept SNMP traffic from a given IP address or
subnet and traffic between the CMTS and the modem should be encrypted via
BPI+ for some minimal security.  (A minor note, the router interface for a
combo unit by CableLabs OSS standards will not respond to SNMP traffic on
its public address by default and almost all of the SNMP traffic will be to
the modem's RFC1918 address.)  What I'm getting at is that the for DOCSIS
(and FTTH and most DSL flavors as well) the service provider has and has
had access to all the settings for a very long time.  What's changed is
that customers wanted to WiFi to be provided by the operator and
importantly supported by the operator.

" I have yet to hear any discussion of the business value of access to WiFi
network password, especially as compared to billing and identity data.
Also, remote management of local networks by definition requires
credentials stored off site from the customer. To the typical customer,
loss of connectivity is much worse than a small chance of sharing the WiFi."

Let me provide something in this regard.  The most common support call
category, by a large number, is the WiFi category.  In excess of 55% of all
support calls (regardless of how the customer describes them) end up being
WiFi issues.  The most common specific call in that category is some
variation of, "I've forgotten my WiFi password and I need to get my new
iPad online."  As a service provider your choices are to say I can reset
your WiFi PSK, which allows the new device to come online but breaks
everything else that was connected, or to allow the support rep and/or
customer to recover the passphrase.  In short, the ability to recover the
passphrase is strongly preferred by end users over resetting it and frankly
is much less expensive for the operator.

I've helped supply this functionality to a large number of operators and in
general the implementations _should_ at a minimum be able to capture the
agent who recovered the passphrase, the time/date, who the agent was on the
phone with, whether the agent correctly verified the identity of the
caller, and if the agent followed all other procedures laid by the service
provider.

Scott Helms



On Thu, Apr 25, 2019 at 8:38 AM James R Cutler 
wrote:

> On Apr 25, 2019, at 8:26 AM, K. Scott Helms 
> wrote:
>
> People are missing the point here.  This is _not_ a Comcast "issue" this
> same data is available to every single cable operator in the US who deploys
> bundled modem/router/APs that follow the CableLabs standard.  They may or
> may not expose the data to their end customers, but it's stored in their
> systems and is often available to their support teams.
>
> http://mibs.cablelabs.com/MIBs/wireless/CLAB-WIFI-MIB-2017-09-07.txt
>
> The same thing applies to most FTTH and xDSL deployments.  They use TR-069
> to collect the data instead of SNMP but the object model is the same.  The
> WiFi MIB above is explicitly built to mimic TR-181 functionality.
>
> Scott Helms
>
>
>
> On Wed, Apr 24, 2019 at 5:48 PM Rich Kulawiec  wrote:
>
>> On Wed, Apr 24, 2019 at 03:13:33PM +, Benjamin Sisco wrote:
>> > The bigger concern should be the cleartext portion of the subject.
>>
>> Yes, and the availability of all this to anyone who hacks Comcast
>> customer support.
>>
>> —rsk
>
>
> I have yet to hear any discussion of the business value of access to WiFi
> network password, especially as compared to billing and identity data.
> Also, remote management of local networks by definition requires
> credentials stored off site from the customer. To the typical customer,
> loss of connectivity is much worse than a small chance of sharing the WiFi.
>
> Narrowing the discussion back to Comcast, credentials for “guest” WiFi
> seem to be more used than purloined SNMP MIB data.
>

Re: Comcast storing WiFi passwords in cleartext?

[K. Scott Helms]

People are missing the point here.  This is _not_ a Comcast "issue" this
same data is available to every single cable operator in the US who deploys
bundled modem/router/APs that follow the CableLabs standard.  They may or
may not expose the data to their end customers, but it's stored in their
systems and is often available to their support teams.

http://mibs.cablelabs.com/MIBs/wireless/CLAB-WIFI-MIB-2017-09-07.txt

The same thing applies to most FTTH and xDSL deployments.  They use TR-069
to collect the data instead of SNMP but the object model is the same.  The
WiFi MIB above is explicitly built to mimic TR-181 functionality.

Scott Helms



On Wed, Apr 24, 2019 at 5:48 PM Rich Kulawiec  wrote:

> On Wed, Apr 24, 2019 at 03:13:33PM +, Benjamin Sisco wrote:
> > The bigger concern should be the cleartext portion of the subject.
>
> Yes, and the availability of all this to anyone who hacks Comcast
> customer support.
>
> ---rsk
>

Re: Comcast storing WiFi passwords in cleartext?

[K. Scott Helms]

While it's correct that it's stored in the vendor proprietary MIB this
information is commonly retrieved from the CableLabs standard MIB and via
TR-181 in DSL and FTTH gear.

I wrote up an answer on the security forum originally refereneced, but for
convenience here is the same text.


The PSK passphrase is (by design) stored in a retrievable format by the
Modem vendor, in this case Arris, but the same standard is supported by
many other modem vendors. In DOCSIS cable modems this is most commonly done
via SNMP against this specific OID:

clabWIFIAccessPointSecurityKeyPassphrase OBJECT-TYPE SYNTAX SnmpAdminString
(SIZE(0..63))
MAX-ACCESS read-create STATUS current DESCRIPTION "This object is defined
in TR-181 Device.WiFi.AccessPoint{i}.Security.KeyPassphrase." REFERENCE
"TR-181 Device Data Model for TR-069." ::=
{clabWIFIAccessPointSecurityEntry 5

This is part of the CableLabs WiFi MIB:

http://mibs.cablelabs.com/MIBs/wireless/CLAB-WIFI-MIB-2017-09-07.txt

Which is is in turn based on the TR-069 sub-standard of TR-181:

https://cwmp-data-models.broadband-forum.org/tr-181-2-11-0.html#D.Device:2.Device.WiFi.AccessPoint
.{i}.Security.KeyPassphrase

http://www.broadband-forum.org/download/TR-181_Issue-2_Amendment-2.pdf

Not only does this apply to cable modems, but many DSL and FTTH endpoints
will also allow the service provider to retrieve your PSK passphrases and a
litany of other settings.

This allows for end users to have their settings backed up in case of a
device having to be replaced or much more commonly for call centers to be
able to retrieve some of the settings, like the pass phrase, when a
customer calls in because they can't remember it.
Scott Helms



On Tue, Apr 23, 2019 at 11:34 PM Luke Guillory 
wrote:

> Yes it's in the router, accessed via the following MIB.
>
>
>
> Name arrisRouterWPAPreSharedKey
> OID  .1.3.6.1.4.1.4115.1.20.1.1.3.26.1.2
> MIB  ARRIS-ROUTER-DEVICE-MIB
> Syntax   OCTET STRING (SIZE (8..64))
> Access   read-write
> Status   current
>
> Descri   Sets the WPA Pre-Shared Key (PSK) used by this service set.  This
>value MUST be either a 64 byte hexadecimal number, OR an 8
> to 63
>character ASCII string.
>
>
> Which returns the following.
>
>
> OID: .1.3.6.1.4.1.4115.1.20.1.1.3.26.1.2.10004
> Value: F2414322EE3D9263
> Type: OctetString
>
> OID: .1.3.6.1.4.1.4115.1.20.1.1.3.26.1.2.10003
> Value: F2414322EE3D9263
> Type: OctetString
>
> OID: .1.3.6.1.4.1.4115.1.20.1.1.3.26.1.2.10002
> Value: F2414322EE3D9263
> Type: OctetString
>
> OID: .1.3.6.1.4.1.4115.1.20.1.1.3.26.1.2.10001
> Value: F2414322EE3D9263
> Type: OctetString
>
>
>
>
>
> Ns
>
>
>
>
>
>
>
> -Original Message-
> From: Peter Beckman [mailto:beck...@angryox.com]
> Sent: Tuesday, April 23, 2019 9:35 PM
> To: Luke Guillory
> Cc: Laurent Dumont; NANOG
> Subject: Re: Comcast storing WiFi passwords in cleartext?
>
> On Tue, 23 Apr 2019, Peter Beckman wrote:
>
> > On Wed, 24 Apr 2019, Luke Guillory wrote:
> >
> >> OP said they logged into their account and went to the security
> >> portion of the portal. So one can assume they're the ISP or I don’t
> >> see the point in asking how Comcast would know the info.
> >
> > It is entirely possible that an account separate and hidden from the
> > customer account would be able to access the administrative controls
> > of the router. It is also plausible that the access does not use a
> > username/password to authenticate but another, hopefully secure method.
> >
> > One could make this access secure by:
> >
> >1. Ensuring any connection originated from Company-controlled IP space
> >2. Username/Password are not provided to the CS agent but is merely a
> >button they press, after properly authenticating themselves as
> well
> >as authenticating the customer, that would pass a one-time use
> >token to access the device
> >3. Every token use was logged and regularly audited
> >4. Keys were regularly and in an automated fashion rotated, maybe even
> >   daily
> >
> > If such precautions are taken, it is their router and it is their
> > service, seems reasonable that Comcast should be able to log into
> > their router and change configs.
>
> ... such that the access of the Wifi Password which is likely stored in
> plain text on the router is accessed by Comcast in a secure manner and not
> stored in plain text in their internal databases.
>
> But I'm guessing probably it's just cached in plain text in their internal
> DBs.
>
> Get your own router if you're worried about your Wifi Password being known
> by Comcast. Or change to WPA2 Enterprise, but I'm guessing that isn't
> supported on the router...
>
> ---
> Peter Beckman  Internet Guy
> beck...@angryox.com
> http://www.angryox.com/
> ---
>

Re: Cellular backup connections

[K. Scott Helms]

I really can't believe I'm going to say this, but this has been a good
SD-WAN use case for us.


Scott Helms



On Fri, Dec 28, 2018 at 2:30 PM Aaron1  wrote:

> On the topic of static ip... as a Net Eng of an ISP, and seeing the pains
> that we have to endure with our static ip customers , I wonder if static ip
> customers actually inadvertently get less optimal treatment than more
> flexible, agile and dynamic ip customers ?
>
> I’m saying that since over the years as I have migrated from one router to
> another, from one technology Ethernet/IP, mpls/ip, it’s more difficult to
> move those static customers subnets around, and sometimes easier just to
> leave them on an old router where they’ve been for years.
>
> Aaron
>
> On Dec 28, 2018, at 12:32 PM, Jared Geiger  wrote:
>
> I found horrible routing with a static IP setup with T-Mobile. The device
> was located in Ashburn, outbound routing would go out via Dallas and
> inbound would come in via Seattle. So ping times and usability was rough.
> Tried it on the west coast and the same problem. T-Mobile support said this
> was by design and they couldn’t change it.
>
> I decided to switch to a regular consumer AT data sim without a static
> IP and set up a small router to initiate a VPN tunnel out to wherever I
> need it. It turns out to be cheaper and reliable for us.
>
> ~Jared Geiger
>
> On Fri, Dec 28, 2018 at 11:53 AM Ryan Wilkins  wrote:
>
>> You mention your connection is 4G.  On T-Mobile 4G is UMTS whereas LTE
>> is, well, LTE.  Are you really on UMTS (which I would expect to have much
>> crazier RTTs and jitter like you report) or did you mean LTE?
>>
>> Ryan
>>
>> > On Dec 28, 2018, at 7:06 AM, Dovid Bender  wrote:
>> >
>> > Hi All,
>> >
>> > I finally got around to setting up a cellular backup device in our new
>> POP. I am currently testing with T-Mobile where the cell signal strength is
>> at 80%. The connection is 4G. When SSH'ing in remotely the connection seems
>> rather slow. Ping times seem to be all over the place (for instance now I
>> am seeing: rtt min/avg/max/mdev = 174.142/336.792/555.574/99.599 ms) . Is
>> that just cellular or is that more related to the provider and the location
>> where I am? I could in theory test with VZ and ATT as well. With Verizon
>> they charge $500.00 just to get a public IP and I want to avoid that if
>> possible.
>> >
>> > Thanks and sorry in advance if this is off topic.
>> >
>> >
>>
>>

Re: Enterprise GPON / Zhone Questions

[K. Scott Helms]

I'd say that any carrier grade GPON gear is way overkill for a LAN and
you're going to have to run single mode fiber to use the consumer grade
ONTs which is a big extra expense as few structured wiring companies do
single mode.  Second, Dasan Zhone is one of the vendors I'd absolutely
avoid and I've worked on numerous GPON OLTs (Adtran TA5000/3000, Calix C7,
E7, E3, and others).  Their configuration is problematic as you've found
out and they have a poor track record in security.

https://www.securityweek.com/over-million-dasan-routers-vulnerable-remote-hacking

Using third party optics is (with all the GPON vendors) a complete crap
shoot.  Sometimes they will work and suddenly a firmware update from the
OLT vendor comes along and they no longer work.  Other times they don't
work at all or are very unreliable.

GPON is a standard, but in North America the vendors have largely not been
forced to do interoperability and it's very lacking.  Compare that to
Europe where the Fritzbox is one of the most popular ONTs.

Finally, as many have said I cannot see any scenario where building GPON
will be as cost effective, reliable, or performant as simply building out a
switched Ethernet network over fiber.


On Tue, Dec 11, 2018 at 10:53 AM Nick Bogle  wrote:

> Hello fellow NANOG members :)
>
> Let me start with a little bit of background, my day job is a Network
> Engineer for a local university where we have primarily a Cisco environment
> from phones to switching to routing, etc. Before my time, we hired a
> contractor to design a GPON LAN system for a new building as a cost saving
> measure (though I am not sure how successful that was).
>
> Either way, the contractor is about to hand the system off to us, and we
> have gone through the training and such, and I feel confident in my ability
> to manage the system, but we have a few questions that the manufacturer of
> our equipment and our contractor didn't really want to answer. We are
> currently using a Dasan Zhone MXK-F1419 with several different downstream
> ONT models (all Zhone).
>
> -We would like to consider use of 3rd party GPON B+ Optics on the
> linecards to add redundancy to the splitter (as the cost of 1st party are
> too high). Does anyone have experience with 3rd party
> vendors/compatibility/stability issues? We were told they theoretically
> should work and just throw a log event, but it hasn't been tested. If so,
> what vendors would you recommend? So far all we've really seen are Ubiquiti
> and Fiberstore optics.
>
> -As GPON is a standard itself, I'm aware interoperability between OLT and
> ONT vendors is heavily limited.. Does anyone have any experience using say,
> Zhone ONT's with a different model OLT, or Zhone ONT's with a different
> model OLT? I've heard word that Zhone ONT's may be able to work with Nokia
> OLT's but it's technically not supported.
>
> -We've already experienced some pretty big stability issues (have replaced
> 1 line card 5 times..), our contractor is saying it's just because we were
> a pretty early adopter of this line and that they've fixed it and fixed
> internal policies to add additional QA and testing before shipping to
> customers. Does anyone have any experience with working with Zhone and
> their overall stability of components?
>
> - Any other thoughts/gotchas/advice for deploying a GPON environment in a
> corporate LAN? (or about deploying a Zhone solution) It's pretty service
> provider oriented, and is incredible noticeable in the CLI.
>
> -Is anyone familiar with Zhone CLI? The apparent lack of any "show"
> configuration commands is infuriating.
>
>
> Feel free to contact me offlist if you have any pertinent info that you
> don't want on the list.
>
> Thanks,
>
> Nick Bogle
> n...@bogle.se
>

Re: Proving Gig Speed

[K. Scott Helms]

> Peering isn't the problem. Proximity to content is.
>
> Netflix, Google, Akamai and a few others have presence in Africa already.
> So those aren't the problem (although for those currently in Africa, not
> all of the services they offer globally are available here - just a few).
>
> A lot of user traffic is not video streaming, so that's where a lot of
> work is required. In particular, cloud and gaming operators are the ones
> causing real pain.
>
> All the peering in the world doesn't help if the latency is well over
> 100ms+. That's what we need to fix.
>
> Mark.
>

Mark,

I agree completely, I'm working on a paper right now for a conference
(waiting on Wireshark to finish with my complex filter at the moment) that
shows what's happening with gaming traffic.  What's really interesting is
how gaming is changing and within the next few years I do expect a lot of
games to move into the remote rendering world.  I've tested several and the
numbers are pretty substantial.  You need to have <=30 ms of latency to
sustain 1080p gaming and obviously jitter and packet loss are also
problematic.  The traffic is also pretty impressive with spikes of over 50
mbps down and sustained averages over 21 mbps.  Upstream traffic isn't any
more of an issue than "normal" online gaming.  Nvidia, Google, and a host
of start ups are all in the mix with a lot of people predicting Sony and
Microsoft will be (or are already) working on pure cloud consoles.

Scott Helms

Re: Proving Gig Speed

[K. Scott Helms]

Mark,

I am glad I don't have your challenges :)

What's the Netflix (or other substantial OTT video provider) situation for
direct peers?  It's pretty easy and cheap for North American operators to
get settlement free peering to Netflix, Amazon, Youtube and others but I
don't know what that looks like in Africa.

Scott Helms

On Wed, Jul 18, 2018 at 10:00 AM Mark Tinka  wrote:

>
>
> On 18/Jul/18 15:41, K. Scott Helms wrote:
>
>
>
> That's why I vastly prefer stats from the actual CDNs and content
> providers that aren't generated by speed tests.  They're generated by
> measuring the actual performance of the service they deliver.  Now, that
> won't prevent burden shifting, but it does get rid of a lot of the problems
> you bring up.  Youtube for example wouldn't rate a video stream as good if
> the packet loss were high because it's actually looking at the bit rate of
> successfully delivered encapsulated video frames I _think_ the same is true
> of Netflix though they also offer a real time test as well which frankly
> isn't as helpful for monitoring but getting a quick test to the Netflix
> node you'd normally use can be nice in some cases.
>
>
> Agreed.
>
> In our market, we've generally not struggled with users and their
> experience for services hosted locally in-country.
>
> So in addition to providing good tools for operators and eyeballs to
> measure experience, the biggest win will come from the content folk and
> CDN's getting their services inside our market.
>
> Mark.
>
>

Re: Proving Gig Speed

[K. Scott Helms]

That seems only to be for direct peers Mike.

On Wed, Jul 18, 2018 at 9:53 AM Mike Hammett  wrote:

> https://isp.google.com
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
>
> From: "K. Scott Helms" 
> To: "Mike Hammett" 
> Cc: "NANOG list" 
> Sent: Wednesday, July 18, 2018 8:45:22 AM
> Subject: Re: Proving Gig Speed
>
>
> Mike,
>
> What portal would that be? Do you have a URL?
>
>
> On Wed, Jul 18, 2018 at 9:25 AM Mike Hammett < na...@ics-il.net > wrote:
>
>
> Check your Google portal for more information as to what Google can do
> with BGP Communities related to reporting.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
>
> From: "K. Scott Helms" < kscott.he...@gmail.com >
> To: "mark tinka" < mark.ti...@seacom.mu >
> Cc: "NANOG list" < nanog@nanog.org >
> Sent: Wednesday, July 18, 2018 7:40:31 AM
> Subject: Re: Proving Gig Speed
>
> Agreed, and it's one of the fundamental problems that a speed test is (and
> can only) measure the speeds from point A to point B (often both inside
> the
> service provider's network) when the customer is concerned with traffic to
> and from point C off in someone else's network altogether. It's one of the
> reasons that I think we have to get more comfortable and more
> collaborative
> with the CDN providers as well as the large sources of traffic. Netflix,
> Youtube, and I'm sure others have their own consumer facing performance
> testing that is _much_ more applicable to most consumers as compared to
> the
> "normal" technician test and measurement approach or even the service
> assurance that you get from normal performance monitoring. What I'd really
> like to see is a way to measure network performance from the CO/head
> end/PoP and also get consumer level reporting from these kinds of
> services. If Google/Netflix/Amazon Video/$others would get on board with
> this idea it would make all our lives simpler.
>
> Providing individual users stats is nice, but if these guys really want to
> improve service it would be great to get aggregate reporting by ASN. You
> can get a rough idea by looking at your overall graph from Google, but
> it's
> lacking a lot of detail and there's no simple way to compare that to a
> head
> end/CO test versus specific end users.
>
> https://www.google.com/get/videoqualityreport/
> https://fast.com/#
>
>
>
> On Wed, Jul 18, 2018 at 8:27 AM Mark Tinka < mark.ti...@seacom.mu >
> wrote:
>
> >
> >
> > On 18/Jul/18 14:00, K. Scott Helms wrote:
> >
> >
> > That's absolutely a concern Mark, but most of the CPE vendors that
> support
> > doing this are providing enough juice to keep up with their max
> > forwarding/routing data rates. I don't see 10 Gbps in residential
> Internet
> > service being normal for quite a long time off even if the port itself
> is
> > capable of 10Gbps. We have this issue today with commercial customers,
> but
> > it's generally not as a much of a problem because the commercial CPE get
> > their usage graphed and the commercial CPE have more capabilities for
> > testing.
> >
> >
> > I suppose the point I was trying to make is when does it stop being
> > feasible to test each and every piece of bandwidth you deliver to a
> > customer? It may very well not be 10Gbps... perhaps it's 2Gbps, or
> 3.2Gbps,
> > or 5.1Gbps... basically, the rabbit hole.
> >
> > Like Saku, I am more interested in other fundamental metrics that could
> > impact throughput such as latency, packet loss and jitter. Bandwidth,
> > itself, is easy to measure with your choice of SNMP poller + 5 minutes.
> But
> > when you're trying to explain to a simple customer buying 100Mbps that a
> > break in your Skype video cannot be diagnosed with a throughput speed
> test,
> > they don't/won't get it.
> >
> > In Africa, for example, customers in only one of our markets are so
> > obsessed with speed tests. But not to speed test servers that are
> > in-country... they want to test servers that sit in Europe, North
> America,
> > South America and Asia-Pac. With the latency averaging between 140ms -
> > 400ms across all of those regions from source, the amount of energy
> spent
> > explaining to customers that there is no way you can saturate your
> > de

Re: Proving Gig Speed

[K. Scott Helms]

Mike,

What portal would that be?  Do you have a URL?

On Wed, Jul 18, 2018 at 9:25 AM Mike Hammett  wrote:

> Check your Google portal for more information as to what Google can do
> with BGP Communities related to reporting.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> ----- Original Message -
>
> From: "K. Scott Helms" 
> To: "mark tinka" 
> Cc: "NANOG list" 
> Sent: Wednesday, July 18, 2018 7:40:31 AM
> Subject: Re: Proving Gig Speed
>
> Agreed, and it's one of the fundamental problems that a speed test is (and
> can only) measure the speeds from point A to point B (often both inside
> the
> service provider's network) when the customer is concerned with traffic to
> and from point C off in someone else's network altogether. It's one of the
> reasons that I think we have to get more comfortable and more
> collaborative
> with the CDN providers as well as the large sources of traffic. Netflix,
> Youtube, and I'm sure others have their own consumer facing performance
> testing that is _much_ more applicable to most consumers as compared to
> the
> "normal" technician test and measurement approach or even the service
> assurance that you get from normal performance monitoring. What I'd really
> like to see is a way to measure network performance from the CO/head
> end/PoP and also get consumer level reporting from these kinds of
> services. If Google/Netflix/Amazon Video/$others would get on board with
> this idea it would make all our lives simpler.
>
> Providing individual users stats is nice, but if these guys really want to
> improve service it would be great to get aggregate reporting by ASN. You
> can get a rough idea by looking at your overall graph from Google, but
> it's
> lacking a lot of detail and there's no simple way to compare that to a
> head
> end/CO test versus specific end users.
>
> https://www.google.com/get/videoqualityreport/
> https://fast.com/#
>
>
>
> On Wed, Jul 18, 2018 at 8:27 AM Mark Tinka  wrote:
>
> >
> >
> > On 18/Jul/18 14:00, K. Scott Helms wrote:
> >
> >
> > That's absolutely a concern Mark, but most of the CPE vendors that
> support
> > doing this are providing enough juice to keep up with their max
> > forwarding/routing data rates. I don't see 10 Gbps in residential
> Internet
> > service being normal for quite a long time off even if the port itself
> is
> > capable of 10Gbps. We have this issue today with commercial customers,
> but
> > it's generally not as a much of a problem because the commercial CPE get
> > their usage graphed and the commercial CPE have more capabilities for
> > testing.
> >
> >
> > I suppose the point I was trying to make is when does it stop being
> > feasible to test each and every piece of bandwidth you deliver to a
> > customer? It may very well not be 10Gbps... perhaps it's 2Gbps, or
> 3.2Gbps,
> > or 5.1Gbps... basically, the rabbit hole.
> >
> > Like Saku, I am more interested in other fundamental metrics that could
> > impact throughput such as latency, packet loss and jitter. Bandwidth,
> > itself, is easy to measure with your choice of SNMP poller + 5 minutes.
> But
> > when you're trying to explain to a simple customer buying 100Mbps that a
> > break in your Skype video cannot be diagnosed with a throughput speed
> test,
> > they don't/won't get it.
> >
> > In Africa, for example, customers in only one of our markets are so
> > obsessed with speed tests. But not to speed test servers that are
> > in-country... they want to test servers that sit in Europe, North
> America,
> > South America and Asia-Pac. With the latency averaging between 140ms -
> > 400ms across all of those regions from source, the amount of energy
> spent
> > explaining to customers that there is no way you can saturate your
> > delivered capacity beyond a couple of Mbps using Ookla and friends is
> > energy I could spend drinking wine and having a medium-rare steak,
> instead.
> >
> > For us, at least, aside from going on a mass education drive in this
> > particular market, the ultimate solution is just getting all that
> content
> > localized in-country or in-region. Once that latency comes down and the
> > resources are available locally, the whole speed test debacle will
> easily
> > fall away, because the source of these speed tests is simply how
> physically
> > far the content is. Is this an easy task - hell no; but slamming your
> head
> > against a wall over and over is no fun either.
> >
> > Mark.
> >
>
>

Re: Proving Gig Speed

[K. Scott Helms]

On Wed, Jul 18, 2018 at 9:01 AM Mark Tinka  wrote:

>
> Personally, I don't think the content networks and CDN's should focus on
> developing yet-another-speed-test-server, because then they are just
> pushing the problem back to the ISP. I believe they should better spend
> their time:
>
>- Delivering as-near-to 100% of all of their services to all regions,
>cities, data centres as they possibly can.
>
>
>- Providing tools for network operators as well as their consumers
>that are biased toward the expected quality of experience, rather than how
>fast their bandwidth is. A 5Gbps link full of packet loss does not a
>service make - but what does that translate into for the type of service
>the content network or CDN is delivering?
>
> Mark.
>
That's why I vastly prefer stats from the actual CDNs and content providers
that aren't generated by speed tests.  They're generated by measuring the
actual performance of the service they deliver.  Now, that won't prevent
burden shifting, but it does get rid of a lot of the problems you bring
up.  Youtube for example wouldn't rate a video stream as good if the packet
loss were high because it's actually looking at the bit rate of
successfully delivered encapsulated video frames I _think_ the same is true
of Netflix though they also offer a real time test as well which frankly
isn't as helpful for monitoring but getting a quick test to the Netflix
node you'd normally use can be nice in some cases.

Re: Proving Gig Speed

[K. Scott Helms]

Agreed, and it's one of the fundamental problems that a speed test is (and
can only) measure the speeds from point A to point B (often both inside the
service provider's network) when the customer is concerned with traffic to
and from point C off in someone else's network altogether.  It's one of the
reasons that I think we have to get more comfortable and more collaborative
with the CDN providers as well as the large sources of traffic.  Netflix,
Youtube, and I'm sure others have their own consumer facing performance
testing that is _much_ more applicable to most consumers as compared to the
"normal" technician test and measurement approach or even the service
assurance that you get from normal performance monitoring.  What I'd really
like to see is a way to measure network performance from the CO/head
end/PoP and also get consumer level reporting from these kinds of
services.  If Google/Netflix/Amazon Video/$others would get on board with
this idea it would make all our lives simpler.

Providing individual users stats is nice, but if these guys really want to
improve service it would be great to get aggregate reporting by ASN.  You
can get a rough idea by looking at your overall graph from Google, but it's
lacking a lot of detail and there's no simple way to compare that to a head
end/CO test versus specific end users.

https://www.google.com/get/videoqualityreport/
https://fast.com/#



On Wed, Jul 18, 2018 at 8:27 AM Mark Tinka  wrote:

>
>
> On 18/Jul/18 14:00, K. Scott Helms wrote:
>
>
> That's absolutely a concern Mark, but most of the CPE vendors that support
> doing this are providing enough juice to keep up with their max
> forwarding/routing data rates.  I don't see 10 Gbps in residential Internet
> service being normal for quite a long time off even if the port itself is
> capable of 10Gbps.  We have this issue today with commercial customers, but
> it's generally not as a much of a problem because the commercial CPE get
> their usage graphed and the commercial CPE have more capabilities for
> testing.
>
>
> I suppose the point I was trying to make is when does it stop being
> feasible to test each and every piece of bandwidth you deliver to a
> customer? It may very well not be 10Gbps... perhaps it's 2Gbps, or 3.2Gbps,
> or 5.1Gbps... basically, the rabbit hole.
>
> Like Saku, I am more interested in other fundamental metrics that could
> impact throughput such as latency, packet loss and jitter. Bandwidth,
> itself, is easy to measure with your choice of SNMP poller + 5 minutes. But
> when you're trying to explain to a simple customer buying 100Mbps that a
> break in your Skype video cannot be diagnosed with a throughput speed test,
> they don't/won't get it.
>
> In Africa, for example, customers in only one of our markets are so
> obsessed with speed tests. But not to speed test servers that are
> in-country... they want to test servers that sit in Europe, North America,
> South America and Asia-Pac. With the latency averaging between 140ms -
> 400ms across all of those regions from source, the amount of energy spent
> explaining to customers that there is no way you can saturate your
> delivered capacity beyond a couple of Mbps using Ookla and friends is
> energy I could spend drinking wine and having a medium-rare steak, instead.
>
> For us, at least, aside from going on a mass education drive in this
> particular market, the ultimate solution is just getting all that content
> localized in-country or in-region. Once that latency comes down and the
> resources are available locally, the whole speed test debacle will easily
> fall away, because the source of these speed tests is simply how physically
> far the content is. Is this an easy task - hell no; but slamming your head
> against a wall over and over is no fun either.
>
> Mark.
>

Re: Proving Gig Speed

[K. Scott Helms]

That's absolutely a concern Mark, but most of the CPE vendors that support
doing this are providing enough juice to keep up with their max
forwarding/routing data rates.  I don't see 10 Gbps in residential Internet
service being normal for quite a long time off even if the port itself is
capable of 10Gbps.  We have this issue today with commercial customers, but
it's generally not as a much of a problem because the commercial CPE get
their usage graphed and the commercial CPE have more capabilities for
testing.

Scott Helms


On Tue, Jul 17, 2018 at 8:11 AM Mark Tinka  wrote:

>
>
> On 17/Jul/18 14:07, K. Scott Helms wrote:
>
>
> That's absolutely true, but I don't see any real alternatives in some
> cases.  I've actually built automated testing into some of the CPE we've
> deployed and that works pretty well for some models but other devices don't
> seem to be able to fill a ~500 mbps link.
>
>
> So what are you going to do when 10Gbps FTTH into the home becomes the
> norm?
>
> Perhaps laptops and servers of the time won't even see this as a rounding
> error :-\...
>
> Mark.
>

Re: Proving Gig Speed

[K. Scott Helms]

That's absolutely true, but I don't see any real alternatives in some
cases.  I've actually built automated testing into some of the CPE we've
deployed and that works pretty well for some models but other devices don't
seem to be able to fill a ~500 mbps link.


On Tue, Jul 17, 2018 at 8:03 AM Mark Tinka  wrote:

>
>
> On 16/Jul/18 22:31, Carlos Alcantar wrote:
>
> > It's a complete rabbit hole...
>
> Couldn't have said it better myself!
>
> Mark.
>

Re: Impacts of Encryption Everywhere (any solution?)

[K. Scott Helms]

Mark,

A couple of things, first that kind of utilization isn't feasible once
penetration rates in dense areas reach certain levels.  There's a reason
that NTT Docomo moved more than 70% of their data traffic to the 3.5 GHz
band and that reason is that there's not (nor will there be) enough
wireless spectrum to meet the needs of everyone with licensed space.  (That
same use case is why all the big North American providers are looking at
CBRS.) Further, 4G/5G is going to have trouble scaling to the kinds of
network demands going forward, again especially in dense areas.  While it's
certainly possible today to stream unicast video over LTE and will (for a
while) even more feasible over 5G the physics simply aren't with the
wireless world.

I'd say that your example of poor DSL performance isn't unique, it happens
in some spots in the US, but in general wired performance has much higher
individual and even higher aggregate capacities *when correctly deployed.  *I
doubt your hotel example is a poor deployment though, it's more likely that
the hotel owners are under paying for both the WAN connection and the WiFi
infrastructure.


On Wed, May 30, 2018 at 1:01 PM Mark Tinka  wrote:

>
>
> On 30/May/18 17:11, McBride, Mack wrote:
>
> > In high density urban areas last mile infrastructure (mostly copper) is
> considerably better than 4G.
> > Localized carrier powered wifi is good as well but it is not and should
> not be confused with 4G.
>
> I think it depends on what it is you're trying to do. If your
> application is linear IPTV streaming into your home, that probably isn't
> a great idea for any kind of non-wired media. On the other hand, in
> South Africa, where I live, it is routine to deliver video streaming
> services (Netflix, Youtube, ShowMax, e.t.c.) to one's home over 4G/LTE,
> to the extent that the service providers have special data plans that
> support these kinds of use-cases.
>
> In South Africa, I generally find wi-fi in the hotels to be pretty bad,
> as the majority of them tend to be on ADSL backhaul, which averages
> between 1Mbps - 4Mbps to support several dozen or more rooms. A few
> hotels have migrated to fibre, but between guessing what last mile
> they're on and how they operate the wi-fi network, I ALWAYS prefer to
> tether my iPhone to my laptop and work when I'm on the road within the
> country. In all major cities, my 3G/4G performs a lot more reliably,
> better and predictably than most cafe, hotel or mall wi-fi. I don't even
> bother when hotels offer their wi-fi vouchers upon check-in.
>
> With my 4G services (Vodacom and MTN), I can average between 30Mbps -
> 55Mbps when tethering, and that's plenty enough for me. I have a decent
> monthly data plan that I don't have to worry about running out. Of
> course, performance isn't as great if you're in a remote part of the
> country, but that's not unique to South Africa.
>
> Mark.
>

Re: Opinions on intent-based networking

[K. Scott Helms]

A substantial percentage, perhaps 100%, of the use case can be accomplished
using micro-segmentation.

On Tue, May 29, 2018 at 2:33 PM LF OD  wrote:

> Been following the articles on "intent-based" networking from Cisco and
> other vendors for a couple years now, and I have a basic grasp of the
> concept of "define your goals/outcomes and automation will make it so", but
> I do not know the practical applications of it or how you are supposed to
> convey your "intent". However, $dayjob is a medium-sized enterprise and
> looking at a potential refresh in the upcoming budget.
>
>
> So... do any of you at ISPs, Cloud Providers, or Enterprises have any
> hands-on experience with "intent-based" networking products? If so, could
> you please provide some info on what caused you to look at it, who did you
> look at, why did you choose whoever you chose, and is it working out for
> you?
>
>
> If you looked at it very closely and decided not to bother with it, I'd
> also be interested on your take. At $dayjob we have been adopting
> automation at the application and workload level, but there hasn't been
> much need for it at the infrastructure level. However, if we can improve
> the services we offer while refreshing that infrastructure, it wouldn't
> hurt - depending on cost of course.
>
>
> Thanks in advance.
>
> LF0D
>

Re: GDPR outside Europe, was Whois vs GDPR, latest news

[K. Scott Helms]

*" PS: For anyone who came into the middle of this argument, my point
isthat if you have no EU nexus, the realistic chances of the EU
takingaction against you round to zero.  If you do have EU nexus, you
betterbehave."*

I'd say this is accurate with a few caveats and most of those won't apply
to NANOG folks.  One, if you or your company is involved in direct
marketing then you'd better pay attention now even if you don't
intentionally market to people in the EU.  Two, if you work on sensitive
PII (by the GDPR definitions) and you may have EU data subjects' PII.
Three, if you or your company are making public statements about GDPR not
applying to you or making false statements publicly about how your opt out
set up is GDPR compliant (when it can't be).

When I first was involved with international contracts we had a series of
meetings with our executives and legal.  The first thing we heard from
legal were things like, "your contracts aren't enforceable in Europe or
Asia".  When we dived into those statements what we found was that was
practically true, because enforcing them required us to go down one of two
(both expensive) pathways.  Establish a corporate identify in all the
places we wanted to do business and then we could more easily sue in the
local court system where our customers were located _or_ we could sue in US
court and then (provided we won) take that US ruling to the local courts
with jurisdiction over the customer in question.  Both were entirely
possible from a legal standpoint, but neither were practical since the cost
of taking either path would greatly exceed the value of the contract in
question.  Instead of doing that we simply set things up so that we can
quickly turn off services and we billed a month in advance rather than post
billing the way we did in North America.

What I'm getting at is that international enforcement of decisions is
expensive and while the EU has a lot of resources, lawyers, and money
they're still going to be prioritizing their "target" selection.  They're
definitely (as we see from the Facebook fine) going after the big, and in
their minds, egregious abusers of privacy.  Unless you or your company is
very large, international in nature, or doing something they'd consider
very abusive then you're not likely to be at the top of that list.  Having
said that, once things shake out and the big fish are all either compliant
or in court then the regulators will start looking down list.

In fairness, the regulators I spoke with emphasized that they're not "head
hunting" (their words) and that don't want to harm companies that might
inadvertently be violating GDPR in small ways.  I expect that many more
warning letters will be sent than actual fines or regulatory actions this
year.


On Thu, May 24, 2018 at 6:31 PM John Levine  wrote:

> In article <0bb31bbb-388d-4832-85dd-30c01c187...@jeffmurphy.org> you
> write:
> >There’s speculation that enforcement could occur via the FTC Privacy
> Shield program.
>
> Privacy Shield is entirely optional. Joining it requires a lot of
> paperwork and a substantial administrative fee.  If you don't do all
> that, it doesn't apply to you.  Please see my previous comment about
> people who think they understand the GDPR vs. people who actually do.
>
> https://www.privacyshield.gov/welcome
>
> Also, Privacy Shield is a retread of the Safe Harbour deal which EU
> courts invalidated in 2015.  Max Schrems, the guy who filed the case
> against Safe Harbour, has filed a similar suit against Privacy Shield,
> with Facebook as the defendant.  I wouldn't bet a lot on Privacy
> Shield lasting any better than Safe Harbour did.
>
>
> https://techcrunch.com/2018/04/13/privacy-shield-now-facing-questions-via-legal-challenge-to-facebook-data-flows/
>
> R's,
> John
>
> PS: For anyone who came into the middle of this argument, my point is
> that if you have no EU nexus, the realistic chances of the EU taking
> action against you round to zero.  If you do have EU nexus, you better
> behave.
>

Re: Whois vs GDPR, latest news

[K. Scott Helms]

 Anne,

While I was re-reading some of the emails last night I realized that I
mischaracterized your description here, *"You may accuse me of being a
lawyer here (and rightly so :-) ), but "in", as in "in the Union" (which is
the actual language) is very much open to interpretation.  In a judicial
system where lawsuits have turned on  - I kid you not - the interpretation
of what a comma meant, I can almost guarantee you that "in the Union" is
going to get interpreted through lawsuits, and it is absolutely not outside
the realm of possibility that a U.S. citizen visiting in the EU will bring
a lawsuit based on something happening with their PII while they were "in
the Union".*

I didn't make it clear that you were suggesting that some would make this
claim rather than you making that claim.  Mea culpa :)

Our counselors made it clear (as did the regulators I was able to ask) that
short term visits weren't intended to be covered *in their opinion.*  There
are and will be many questions that won't be fully answered until
adjudicated or more precise language is used to make the meaning clear.
 Juhan Lepassaar (Head of VP Ansip Cabinet, European Commission) was one of
the speakers and we were able to ask questions of him.  It looks like the
video of one of the presentations I was at is now publicly available and I
encourage those with questions to watch it.

https://www.rsaconference.com/speakers/juhan-lepassaar

*" Actually, GDPR specifically requires processors to include statements of
compliance right in their contracts;  we also strongly recommend that
controllers insist on indemnification clauses in their contracts with
processors, because if the processor screws up and there is a breach, the
_controller_ can also be held liable, and the financial penalties in GDPR
are very stiff."*

Yep, this is better (clearer) wording than what I used and is absolutely
correct.



On Thu, May 24, 2018 at 10:21 AM Anne P. Mitchell Esq. <amitch...@isipp.com>
wrote:

>
>
> > On May 23, 2018, at 7:18 PM, K. Scott Helms <kscott.he...@gmail.com>
> wrote:
> >
> > Anything that can tie back to an individual data subject is PII, that
> means email addresses, names in combination with addresses or phone
> numbers, finger prints, or even insufficiently abstracted internal ID
> numbers/codes.
>
> Don't forget IP addresses, as part of the wonderfully vague "online
> identifiers".
>
> > Notice I didn't say EU citizen there, that's because the law and
> regulations (GDPR consists of both) intentionally cover any natural person
> in any of the 28 EU nations including the citizens of non-EU nations.
> >  I don't go as far as I think Anne was suggesting, in that someone in EU
> airspace who sent an email or made a purchase is now suddenly an EU data
> subject.
>
> You may accuse me of being a lawyer here (and rightly so :-) ), but "in",
> as in "in the Union" (which is the actual language) is very much open to
> interpretation.  In a judicial system where lawsuits have turned on  - I
> kid you not - the interpretation of what a comma meant, I can almost
> guarantee you that "in the Union" is going to get interpreted through
> lawsuits, and it is absolutely not outside the realm of possibility that a
> U.S. citizen visiting in the EU will bring a lawsuit based on something
> happening with their PII while they were "in the Union".
>
> > Any company that is covered by the GDPR must be extremely careful that
> any company they do business with is also compliant if that company will
> have access or act as a data processor.  That means that if you are a US
> company that has US only customers, but some of your customers have
> employees that are US citizens but who live in an EU nation then they are
> bound to only use providers that are GDPR compliant.  Now, this will result
> in contractual disputes and/or loss of business rather than having EU
> regulators fine your company directly.  The end result is that many many
> many companies that don't sell or market to the EU are finding themselves
> needing to comply in the same way that companies that sell services to
> medical companies often have to follow HIPAA  (and be audited) even though
> they provide medical services themselves.
> >
>
> Actually, GDPR specifically requires processors to include statements of
> compliance right in their contracts;  we also strongly recommend that
> controllers insist on indemnification clauses in their contracts with
> processors, because if the processor screws up and there is a breach, the
> _controller_ can also be held liable, and the financial penalties in GDPR
> are very stiff.
>
> Anne
>
> Anne P. Mitchell,
> Attorney at Law
> CEO/President,
>

Re: Whois vs GDPR, latest news

[K. Scott Helms]

Anne,

Yep, if you're doing a decent job around securing data then you don't have
much to be worried about on that side of things.  The problem for most
companies is that GDPR isn't really a security law, it's a privacy law (and
set of regulations).  That's where it's hard because there are a limited
number of ways you can, from the EU's standpoint, lawfully process
someone's PII.  Things like opting out and blanket agreements to use all of
someone's data for any reason a company may want are specifically
prohibited.  Even companies that don't intentionally sell into the EU (or
the UK) can find themselves dealing with this if they have customers with
employees in the EU.

On Wed, May 23, 2018 at 12:29 PM, Anne P. Mitchell Esq.  wrote:

>
>
> > On May 23, 2018, at 10:21 AM, Daniel Brisson  wrote:
> >
> >> Also, don't forget the private right of action.  Anyone can file
> anything in the U.S. courts... you  may get it dismissed (although then
> again you may not) but either way, it's going to be time and money out of
> your pocket fighting it.  MUCH better to just get compliant than to end up
> a test case.
> >
> > Isn't "better" a factor of how much it costs to become compliant with
> GPDR?  I'm no expert, but some of the things I've heard sounded not trivial
> to implement (read potentially BIG investment).
> >
> > -dan
>
> In our experience, orgs that are already following all industry best
> practices are, generally, at least 70% of the way to becoming compliant
> already.   Where it can get expensive for the ones who aren't is in
> hardening their systems to provide for better security/privacy.  U.S.
> companies are used to being able to drink at the firehose of data that is
> collected here in the U.S., and use it however they want.. this is the real
> major change.  I suppose you could say it's expensive in that it is
> reducing the ways they can monetize that data.
>
> Anne
>
> Anne P. Mitchell,
> Attorney at Law
> CEO/President,
> SuretyMail Email Reputation Certification and Inbox Delivery Assistance
> GDPR Compliance Consultant
> GDPR Compliance Certification
> http://www.SuretyMail.com/
> http://www.SuretyMail.eu/
>
> Attorney at Law / Legislative Consultant
> Author: Section 6 of the CAN-SPAM Act of 2003 (the Federal anti-spam law)
> Author: The Email Deliverability Handbook
> Legal Counsel: The CyberGreen Institute
> Legal Counsel: The Earth Law Center
> Member, California Bar Cyberspace Law Committee
> Member, Colorado Cybersecurity Consortium
> Member, Board of Directors, Asilomar Microcomputer Workshop
> Member, Advisory Board, Cause for Awareness
> Member, Elevations Credit Union Member Council
> Former Chair, Asilomar Microcomputer Workshop
> Ret. Professor of Law, Lincoln Law School of San Jose
>
> Available for consultations by special arrangement.
> amitch...@isipp.com | @AnnePMitchell
> Facebook/AnnePMitchell  | LinkedIn/in/annemitchell
>
>

Re: Whois vs GDPR, latest news

[K. Scott Helms]

Yeah, that's not accurate.  US organizations sue EU organizations in US
courts (and vice versus) on a regular basis but have EU courts collect the
damages.  Congress can carve out an exemption, but I haven't heard of an
effort in that direction getting started yet.  In the absence of a
legislative exemption the EU regulators can absolutely sue a US entity in
US civil courts and get a ruling based on EU laws and regulations.

Here's a completely unrelated civil case, on libel, that references the
bilateral enforcement and how NY state carved out an exemption.

https://www.npr.org/sections/parallels/2015/03/21/394273902/on-libel-and-the-law-u-s-and-u-k-go-separate-ways

Scott Helms

http://twitter.com/kscotthelms


On Wed, May 23, 2018 at 11:56 AM, Owen DeLong  wrote:

> Not really. If you don’t offer services to EU persons, then you are right.
> However, due to treaties signed by the US and other countries, many places
> outside the EU are subject to GDPR overreach.
>
> Owen
>
>
> > On May 23, 2018, at 05:36, Mike Hammett  wrote:
> >
> > If you don't have operations in the EU, you can not so politely tell the
> EU to piss off.
> >
> >
> >
> >
> > -
> > Mike Hammett
> > Intelligent Computing Solutions
> > http://www.ics-il.com
> >
> > Midwest-IX
> > http://www.midwest-ix.com
> >
> > - Original Message -
> >
> > From: "Matthew Kaufman" 
> > To: "Fletcher Kittredge" 
> > Cc: "NANOG list" 
> > Sent: Monday, May 21, 2018 8:07:15 PM
> > Subject: Re: Whois vs GDPR, latest news
> >
> >> On Mon, May 21, 2018 at 1:56 PM Fletcher Kittredge 
> wrote:
> >>
> >> What about my right to not have this crap on NANOG?
> >>
> >
> >
> > What about the likely truth that if anyone from Europe mails the list,
> then
> > every mail server operator with subscribers to the list must follow the
> > GDPR Article 14 notification requirements, as the few exceptions appear
> to
> > not apply (unless you’re just running an archive).
> >
> > Matthew
> >
>
>

Re: Whois vs GDPR, latest news

[K. Scott Helms]

Of course not, but do you really want to be sued?  Even if the US courts
decline to accept GDPR cases, which is not at all a given since we have a
long history of bilateral enforcement, it costs money to deal with and I
don't want to worry that I'm going to fly one day to a country that will
enforce civil penalties.

While I don't tell most people or companies to worry if they only do
business in the US I also don't think it's a good idea to simply thumb your
nose at the EU regulators.  Some North American direct marketing and data
collection firms are definitely going to get a rude, and expensive,
awakening despite not having any EU operations.

On Wed, May 23, 2018 at 8:49 AM, Mike Hammett <na...@ics-il.net> wrote:

> *shrugs* Me hurting the EU's feelings is rather low on the list of things
> I care about.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> ----- Original Message -
>
> From: "K. Scott Helms" <kscotthe...@gmail.com>
> To: "Mike Hammett" <na...@ics-il.net>
> Cc: "NANOG list" <nanog@nanog.org>
> Sent: Wednesday, May 23, 2018 7:46:19 AM
> Subject: Re: Whois vs GDPR, latest news
>
>
> Sadly this isn't true. While I doubt the EU regulators are going to come
> head hunting for companies any time soon they do have mechanisms in place
> to sanction companies who don't do business in the EU and the scope is
> clearly intended to reach where ever the data of EU natural persons is
> being held.
>
>
> https://gdpr-info.eu/art-3-gdpr/
>
>
>
> I asked one of the EU regulators at RSA how they intended to enforce GDPR
> violations on businesses that don't operate in their jurisdiction and
> without hesitation he told me they'd use civil courts to sue the offending
> companies.
>
>
> On Wed, May 23, 2018 at 8:36 AM, Mike Hammett < na...@ics-il.net > wrote:
>
>
> If you don't have operations in the EU, you can not so politely tell the
> EU to piss off.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
>
> From: "Matthew Kaufman" < matt...@matthew.at >
> To: "Fletcher Kittredge" < fkitt...@gwi.net >
> Cc: "NANOG list" < nanog@nanog.org >
> Sent: Monday, May 21, 2018 8:07:15 PM
> Subject: Re: Whois vs GDPR, latest news
>
>
>
> On Mon, May 21, 2018 at 1:56 PM Fletcher Kittredge < fkitt...@gwi.net >
> wrote:
>
> > What about my right to not have this crap on NANOG?
> >
>
>
> What about the likely truth that if anyone from Europe mails the list,
> then
> every mail server operator with subscribers to the list must follow the
> GDPR Article 14 notification requirements, as the few exceptions appear to
> not apply (unless you’re just running an archive).
>
> Matthew
>
>
>
>
>
>

Re: Whois vs GDPR, latest news

[K. Scott Helms]

Sadly this isn't true.  While I doubt the EU regulators are going to come
head hunting for companies any time soon they do have mechanisms in place
to sanction companies who don't do business in the EU and the scope is
clearly intended to reach where ever the data of EU natural persons is
being held.

https://gdpr-info.eu/art-3-gdpr/

I asked one of the EU regulators at RSA how they intended to enforce GDPR
violations on businesses that don't operate in their jurisdiction and
without hesitation he told me they'd use civil courts to sue the offending
companies.

On Wed, May 23, 2018 at 8:36 AM, Mike Hammett  wrote:

> If you don't have operations in the EU, you can not so politely tell the
> EU to piss off.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
>
> From: "Matthew Kaufman" 
> To: "Fletcher Kittredge" 
> Cc: "NANOG list" 
> Sent: Monday, May 21, 2018 8:07:15 PM
> Subject: Re: Whois vs GDPR, latest news
>
> On Mon, May 21, 2018 at 1:56 PM Fletcher Kittredge 
> wrote:
>
> > What about my right to not have this crap on NANOG?
> >
>
>
> What about the likely truth that if anyone from Europe mails the list,
> then
> every mail server operator with subscribers to the list must follow the
> GDPR Article 14 notification requirements, as the few exceptions appear to
> not apply (unless you’re just running an archive).
>
> Matthew
>
>

Re: New Active Exploit: memcached on port 11211 UDP & TCP being exploited for reflection attacks

[K. Scott Helms]

They use separate service flows and layer 3 interfaces (usually) in DOCSIS
networks but they often use the same DNS infrastructure which is why I
piped up.

Scott Helms


On Mar 2, 2018 4:46 PM, "Michel 'ic' Luczak" <li...@benappy.com> wrote:

The ones I know do so on private VLANs (or ATM circuits on DSL) so anyway
unrelated to any client’s address space. Also, french triple play ISPs use
RFC1918 space for IPTV but again isolated of any customer network so
doesn’t really matter.

> On 2 Mar 2018, at 22:18, K. Scott Helms <kscotthe...@gmail.com> wrote:
>
> I won't comment on the sanity of doing so, but _many_ service providers
use
> EMTAs, ATAs, and other voice devices over RFC1918 space back to their
core.
>

Re: New Active Exploit: memcached on port 11211 UDP & TCP being exploited for reflection attacks

[K. Scott Helms]

I won't comment on the sanity of doing so, but _many_ service providers use
EMTAs, ATAs, and other voice devices over RFC1918 space back to their core.

On Fri, Mar 2, 2018 at 4:11 PM, Mark Andrews  wrote:

> Are you insane. ISPs should never use RFC 1918 addresses for stuff that
> talks to their customers.  They have no way of knowing which addresses the
> customers are using.
>
> Traffic from RFC 1918 addresses should be dropped by any sane border
> router which all routers connecting to a ISP are.
>
> --
> Mark Andrews
>
> > On 2 Mar 2018, at 22:49, Bjørn Mork  wrote:
> >
> > Owen DeLong  writes:
> >
> >> I don’t agree that making RFC-1918 limitations a default in any daemon
> makes any
> >> sense whatsoever.
> >
> > +1
> >
> > One of the more annoying anti-features I know of in this regard is the
> > dnsmasq rebind "protection".  It claims to protect web browsers on the
> > LAN against DNS rebind attacks.  But the implementation does not
> > consider which adresses are used on the LAN at all.  It simply blocks
> > any A record pointing to an RFC1918 address, making a few bogus
> > assumptions:
> > - IPv4 LAN addresses are selected from RFC1918
> > - RFC1918 addresses are never used on the WAN side of a CPE
> > - Noone use IPv6 on their LAN
> >
> > It's hard to know how many users have been bitten by the first
> > one. You'd have to depend on this rebind "protection" in the first
> > place, and that would be stupid.
> >
> > But the second assumption regularily bites end users when their ISP
> > provides some ISP internal service using RFC1918 addresses.  Which of
> course
> > is fine.
> >
> > The anti-feature has been enabled by default in OpenWrt for a long time,
> > ref https://wiki.openwrt.org/doc/uci/dhcp#all_options , which means that
> > there is a large user base having this enabled without knowing.
> >
> >> First, there are plenty of LANs out there that don’t use RFC-1918.
> >>
> >> Second, RFC-1918 doesn’t apply to IPv6 at all,
> >
> > Could you try to explain that to the OpenWrt guys?  Thanks
> >
> >
> >
> > Bjørn
>
>

Re: Opensource SNMP Trap Receivers ???

[K. Scott Helms]

Matthew,

Sadly, open source + SNMP traps != simple.

The simplest option that I've personally used in the past is SNMPTT with
Nagios.

https://paulgporter.net/2013/09/16/nagios-snmp-traps/

http://www.snmptt.org/docs/snmptt.shtml

The main problem is that SNMP traps, like most of SNMP, aren't simple
despite the name.  Having said that it can be done especially if the gear
doesn't change too often.

Scott Helms

On Tue, Feb 13, 2018 at 7:44 AM, Matthew Huff  wrote:

> We are retiring a legacy SNMP system and are looking for a simple,
> opensource SNMP trap receiver/alerting system. We aren't looking for a full
> SNMP system, just something that will receive snmp traps and email/alert
> based on them.
>
> 1) Looking for something off the shelf, not a development project
> 2) Opensource or low cost
> 3) SNMP MIB compiler
>
> Any suggestions?
>
> 
> Matthew Huff | 1 Manhattanville Rd
> Director of Operations   | Purchase, NY 10577
> OTA Management LLC   | Phone: 914-460-4039
>
>
>

Re: Blockchain and Networking

[K. Scott Helms]

That sounds like giving up an awful lot of utility for a (at least in most
places) something that's an exceedingly rare corner case.  The other
problem is that even if the record is immutable there's nothing that
prevents governments from being coercive in other ways.  At the end of the
day there's no technology that prevents authoritarian governments from
behaving badly.

On Tue, Jan 23, 2018 at 6:27 PM, Jimmy Hess  wrote:

> On Tue, Jan 23, 2018 at 9:39 AM, John R. Levine  wrote:
>
> >
> > the problem isn't keeping the database, it's figuring out who can make
> > authoritative statements about each block of IP addresses.
>
> That is an inherently hierarchical question since all IP blocks originally
> > trace back to allocations from IANA.
> >
>
> Well;  It's a hierarchical question only because the current registration
> scheme is defined in
> a hierarchical manner.  If  BGP were being designed today,  we could
> choose  256-Bit  AS numbers,
> and allow  each mined or staked block to yield a block of AS numbers
> prepended by some
> random previously-unused 128-bit GUID.
>
> However,  a blockchain could also be used to allow an authority to make a
> statement representing
> a resource that can be made a non-withdrawable statement ---  in other
> words,  the authority's role
> or job in the registration process is to originate the registration,  and
> after that is done:
> their authoritative statement is accepted ONCE per resource.
>
> The registration is permanent ---  the authority has no ongoing role and no
> ability to later make
> a new conflicting statement about that same resource,   and   the
> authority  has  no operational role
> except to originate new registrations.
>
> This would mean that a foreign government could not coerce the authority
> to  "cancel"  or mangle
> a registration to meet a political or adversarial objective of disrupting
> the ability to co-ordinate networks,
> since the  number registry is an authority of  limited power:  not an
> authority of complete power.
>
>
> We can have arguments about the best way to document the chain of
> > ownership, and conspiracy theories about how the evil RIRs are planning
> to
> > steal our precious bodily flu^W^WIPs, but "put it in a blockchain!"
> > Puhleeze.
> > R's,
> > John
> >
>
> --
> -JH
>

Re: Broadcast television in an IP world

[K. Scott Helms]

Mike,

While that's true today it's changing rapidly.  Linear viewership is,
depending on your take on things, either in the beginning or the middle of
it's long tail phase.  You're right in that we'll have people using linear
viewing habits for a long time, but that model only looks sustainable over
the long term for either very large MSOs, the digital satellite operators,
and OTT offerings that offer a similar experience.  There's very little
investing in efficiencies for linear content as this point, other than how
it gets replaced.  Part of the change is technical, part generational
changes, and part overreach on the part of some of the content owners.

Scott Helms

On Tue, Nov 21, 2017 at 2:08 PM, Mike Hammett  wrote:

> I'm not doubting OTT is popular. There's just an awful lot of people that
> have zero interest (or ability) to use OTT. They will continue to consume
> entertainment linearly, regardless of the mechanism used to deliver it.
>
>
> People in NANOG often forget that most people aren't like us. Heck, most
> people in NANOG forget that not every network is like their network.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
>
> From: "Baldur Norddahl" 
> To: nanog@nanog.org
> Sent: Tuesday, November 21, 2017 12:46:43 PM
> Subject: Re: Broadcast television in an IP world
>
> I am not going to guess on a timeframe. I would like to point out that
> the youth ignore TV. They no longer have TVs on their rooms. It is all
> on smartphones or tablets these days. Even with the family in a living
> room, everyone might be sitting with their own device doing their own
> thing.
>
> We have a significant share of the customers that have no other TV than
> OTT streaming. Myself included. Here (Denmark) almost all TV channels
> are available as OTT streaming. The free national broadcast TV is also
> available for streaming (for free).
>
> With an Apple TV you can do all the same things that you can do with
> OTA, cable or satellite. Cheaper (*) and more convenient too. Far from
> everyone has discovered this yet, but since we cater to people that are
> cable cutters, a larger than usual share of our customers is doing
> exactly this.
>
> (*) I believe the OTT solutions are cheaper as long you do not want a
> lot of sport programming. If you do want sport I believe it is more
> expensive but you also have more options and content available.
>
> Regards,
>
> Baldur
>
>
> Den 21/11/2017 kl. 17.58 skrev Mike Hammett:
> > of the TV they use... through you. That doesn't count OTA, cable,
> satellite, etc.
> >
> > It won't change significantly any time soon. I know things are changing,
> but it'll still take five or ten years for those changes to significantly
> change traffic patterns.
> >
> >
> >
> >
> > -
> > Mike Hammett
> > Intelligent Computing Solutions
> > http://www.ics-il.com
> >
> > Midwest-IX
> > http://www.midwest-ix.com
> >
> > - Original Message -
> >
> > From: "Baldur Norddahl" 
> > To: nanog@nanog.org
> > Sent: Tuesday, November 21, 2017 10:52:09 AM
> > Subject: Re: Broadcast television in an IP world
> >
> > Den 21. nov. 2017 16.20 skrev "Mike Hammett" :
> >
> > Unicasting what everyone watches live on a random evening would use
> > significantly more bandwidth than Game of Thrones or whatever OTT drop.
> > Magnitudes more. It wouldn't even be in the same ballpark.
> >
> >
> >
> > I agree as of this moment however that will change. Also note that our
> > customers do 100% of their TV as unicast OTT because that is the only
> thing
> > we offer. This does not cause nearly as much problems as you would
> expect.
> >
>
>
>

Re: Broadcast television in an IP world

[K. Scott Helms]

Luke,

I think I understand your example but the local broadcaster won't usually
(ever?) have the rights to retransmit the Super Bowl over IP.

Having said that, what you're describing is exactly what happens already
(without multicast) via multiple CDNs.  Multicast across the internet isn't
feasible (economically) today.  Multicast inside of an organization
certainly is and is very common.  Having said that, even popular content is
surprisingly sparse (when we look at flows) and even inside of edge
networks (DOCSIS, FTTH, xDSL, etc) it can be surprisingly challenging to
make the math work.  As soon as someone wants to pause the "big game" or
flips to another channel you now have to move their flow to unicast.  Even
when lots of people are watching the same event the economics aren't as
compelling as they might appear initially.


On Tue, Nov 21, 2017 at 9:29 AM, Luke Guillory <lguill...@reservetele.com>
wrote:

> The comment I was originally replying to was the following. I’ve said edge
> resources, nothing about WAN.
>
> The content provider (lets say local TV station that broadcasts the
>
> Superbowl) can just unicast to the ISP a single stream, and give the
>
> ISPs some pizza sized box (lets call it an "Appliance") and that box
>
> then provides unicast delivery to each customer watching the Superbowl.
>
>
>
>
>
> *Sent from my iPhone*
>
> On Nov 21, 2017, at 8:22 AM, K. Scott Helms <kscotthe...@gmail.com> wrote:
>
> It's not helpful for saving resources in DOCSIS (nor any other) edge
> networks.  The economics mean that, as bits get sold in the US and many
> other places, it won't be in the foreseeable future.  Customers care about
> popular video sources.  Popular content sources have CDNs with local nodes
> and/or direct (low cost) connections to their CDN.  That's far more
> efficient than allowing multicast across WAN links.
>
> K. Scott Helms
>
>
>
> Luke Guillory
> Vice President – Technology and Innovation
>
>
> <http://www.rtconline.com>
> Tel: 985.536.1212 <(985)%20536-1212>
> Fax: 985.536.0300 <(985)%20536-0300>
> Email: lguill...@reservetele.com
> Web: www.rtconline.com
> Reserve Telecommunications
> 100 RTC Dr
> <https://maps.google.com/?q=100+RTC+Dr+%0D+Reserve,+LA+70084=gmail=g>
> Reserve, LA 70084
>
>
>
>
>
>
> *Disclaimer:*
> The information transmitted, including attachments, is intended only for
> the person(s) or entity to which it is addressed and may contain
> confidential and/or privileged material which should not disseminate,
> distribute or be copied. Please notify Luke Guillory immediately by
> e-mail if you have received this e-mail by mistake and delete this e-mail
> from your system. E-mail transmission cannot be guaranteed to be secure or
> error-free as information could be intercepted, corrupted, lost, destroyed,
> arrive late or incomplete, or contain viruses. Luke Guillory therefore
> does not accept liability for any errors or omissions in the contents of
> this message, which arise as a result of e-mail transmission.
>
> On Tue, Nov 21, 2017 at 8:58 AM, Luke Guillory <lguill...@reservetele.com>
> wrote:
>
>> I’m not paying anything for local resources with regards to local edge
>> delivery, that’s capital expenditures not MRCs.
>>
>> Our edge networks aren’t unlimited or free, so while it’s not costing me
>> on the transit side there still are cost in terms of upgrades and so on.
>>
>> My point is that In some networks such as docsis conserving edge
>> resources can be helped with multicast.
>>
>>
>>
>> Sent from my iPhone
>>
>> On Nov 21, 2017, at 4:12 AM, Baldur Norddahl <baldur.nordd...@gmail.com
>> <mailto:baldur.nordd...@gmail.com>> wrote:
>>
>> Den 21. nov. 2017 00.42 skrev "Luke Guillory" <lguill...@reservetele.com
>> <mailto:lguill...@reservetele.com>>:
>>
>> Why would an ISP not want to conserve edge resources? If I’m doing iptv
>> I’m
>> better off doing multicast which would conserve loads of BW for something
>> popular like the Super Bowl. Especially if I’m doing this over docsis.
>>
>>
>>
>> You pay for 95th percentile. If that is decided by everyone watching Game
>> of Thrones one day, then using the same resources for Super Bowl the next
>> day will be for free.
>>
>>
>>
>>
>> Luke Guillory
>> Vice President – Technology and Innovation
>>
>>
>> [cid:imagef9b835.JPG@242ea556.429501f5] <http://www.rtconline.com
>> >
>>
>> Tel:985.536.1212
>> Fax:985.536.0300
>> Email:  lguill...@reservetele.c

Re: Broadcast television in an IP world

[K. Scott Helms]

It's not helpful for saving resources in DOCSIS (nor any other) edge
networks.  The economics mean that, as bits get sold in the US and many
other places, it won't be in the foreseeable future.  Customers care about
popular video sources.  Popular content sources have CDNs with local nodes
and/or direct (low cost) connections to their CDN.  That's far more
efficient than allowing multicast across WAN links.

K. Scott Helms

On Tue, Nov 21, 2017 at 8:58 AM, Luke Guillory <lguill...@reservetele.com>
wrote:

> I’m not paying anything for local resources with regards to local edge
> delivery, that’s capital expenditures not MRCs.
>
> Our edge networks aren’t unlimited or free, so while it’s not costing me
> on the transit side there still are cost in terms of upgrades and so on.
>
> My point is that In some networks such as docsis conserving edge resources
> can be helped with multicast.
>
>
>
> Sent from my iPhone
>
> On Nov 21, 2017, at 4:12 AM, Baldur Norddahl <baldur.nordd...@gmail.com<
> mailto:baldur.nordd...@gmail.com>> wrote:
>
> Den 21. nov. 2017 00.42 skrev "Luke Guillory" <lguill...@reservetele.com<
> mailto:lguill...@reservetele.com>>:
>
> Why would an ISP not want to conserve edge resources? If I’m doing iptv I’m
> better off doing multicast which would conserve loads of BW for something
> popular like the Super Bowl. Especially if I’m doing this over docsis.
>
>
>
> You pay for 95th percentile. If that is decided by everyone watching Game
> of Thrones one day, then using the same resources for Super Bowl the next
> day will be for free.
>
>
>
>
> Luke Guillory
> Vice President – Technology and Innovation
>
>
> [cid:imagef9b835.JPG@242ea556.429501f5] <http://www.rtconline.com>
>
> Tel:985.536.1212
> Fax:985.536.0300
> Email:  lguill...@reservetele.com
> Web:www.rtconline.com
>
> Reserve Telecommunications
> 100 RTC Dr
> Reserve, LA 70084
>
>
>
>
>
> Disclaimer:
> The information transmitted, including attachments, is intended only for
> the person(s) or entity to which it is addressed and may contain
> confidential and/or privileged material which should not disseminate,
> distribute or be copied. Please notify Luke Guillory immediately by e-mail
> if you have received this e-mail by mistake and delete this e-mail from
> your system. E-mail transmission cannot be guaranteed to be secure or
> error-free as information could be intercepted, corrupted, lost, destroyed,
> arrive late or incomplete, or contain viruses. Luke Guillory therefore does
> not accept liability for any errors or omissions in the contents of this
> message, which arise as a result of e-mail transmission.
>
>