Re: junos config commit question

[Warren Kumari]

On Fri, Feb 11, 2022 at 5:58 PM Jon Lewis  wrote:

> On Fri, 11 Feb 2022, Lyndon Nerenberg (VE7TFX/VE6BBM) wrote:
>
> > On an EX4300 switch running JunOS 14.1 let's imagine I typed
> >
> >   config
> >   delete interfaces
> >
> > before coming to my senses.  How am I supposed to back out of that
> > mess?  For the life of me, after a week of reading the 3000 page
> > reference manual, and endless DuckDuckGoing, I cannot see a simple
> > way of just abandoning the commit.  I've got to be missing something
> > stunningly obvious here because it's unthinkable that this functionality
> > doesn't exist.  Help?!?
>
> What would you say if I told you a coworker once did exactly that, and did
> commit and-quit...and it had to be fixed by another coworker getting to it
> via OOB console and doing the rollback?  :)
>
> top [not necessary in your case, if you never left top]
> rollback 0
> quit
>
> Also, get into the habit of never doing a commit without first doing
> top
> show | compare
> so you can see what your change is actually doing to the whole config.


My muscle memory includes:
{ some changes }
top
show | compare
commit confirmed 5
{flip over the little electronic egg timer thingie that lives next to my
keyboard, so that it beeps after 3 minutes...wait... wait... press enter a
few times to make sure I haven't screwed myself...}
commit

If I skip the egg timer, then I *will* forget, and it will automatically
roll back. One of my largest annoyances with the Juniper CLI (other than
the fact that it won't format large numbers into a human readable format in
things like 'monitor interface traffic') is that it beeps the terminal
*after* it times out the commit.

Gee, thanks for letting me know you just blew away all of my changes...
couldn't you have done that 1 minute before automatically reverting?!!!


W




> i.e. if you did a show | compare at the top of the config and saw the
> entire interfaces section of the config was "removed" in the resulting
> config diff, you probably wouldn't commit.
>
> --
>   Jon Lewis, MCP :)   |  I route
>   StackPath, Sr. Neteng   |  therefore you are
> _ http://www.lewis.org/~jlewis/pgp for PGP public key_
>
-- 
Perhaps they really do strive for incomprehensibility in their specs.
After all, when the liturgy was in Latin, the laity knew their place.
-- Michael Padlipsky

Re: junos config commit question

[Jason Biel]

My first question is how are you running 14 code on that hardware??

On Fri, Feb 11, 2022 at 20:12 Lyndon Nerenberg (VE7TFX/VE6BBM) <
lyn...@orthanc.ca> wrote:

> Nick Suan via NANOG writes:
> > I was actually interested to see if the EX series would let me do this,
> and i
> > t turns out that if STP is enabled on any of the switch interfaces, it
> won't:
> > tevruden@core-02# commit check
> > [edit protocols rstp]
> >   'interface'
> > XSTP : Interface ge-0/0/0.0 is not enabled for Ethernet Switching
> > error: configuration check-out failed
>
> Do you have any rstp-specific overrides in your config? E.g. we
> have things like this in some of ours:
>
>   rstp {
>   interface ge-0/0/45 {
>   cost 1000;
>   mode point-to-point;
>   }
>   interface ge-1/0/45 {
>   cost 1000;
>   mode point-to-point;
>   }
>   interface ae4;
>   bpdu-block-on-edge;
>   }
>
> With the interfaces gone I would expect the commit check to fail.
>
> --lyndon
>
-- 
Jason

Re: junos config commit question

[Lyndon Nerenberg (VE7TFX/VE6BBM)]

Nick Suan via NANOG writes:
> I was actually interested to see if the EX series would let me do this, and i
> t turns out that if STP is enabled on any of the switch interfaces, it won't:
> tevruden@core-02# commit check 
> [edit protocols rstp]
>   'interface'
> XSTP : Interface ge-0/0/0.0 is not enabled for Ethernet Switching
> error: configuration check-out failed

Do you have any rstp-specific overrides in your config? E.g. we
have things like this in some of ours:

  rstp {
  interface ge-0/0/45 {
  cost 1000;
  mode point-to-point;
  }
  interface ge-1/0/45 {
  cost 1000;
  mode point-to-point;
  }
  interface ae4;
  bpdu-block-on-edge;
  }

With the interfaces gone I would expect the commit check to fail.

--lyndon

RE: New minimum speed for US broadband connections

[Travis Garrison]

In my location, I can get 1.5M from CenturyLink. That is the only hardwired 
option. Typical speeds was around 700K. I spent the money and installed my own 
180ft tower and a microwave connection to a bigger town that I could get a 
fiber circuit at. Now we have linked up several other smaller towns through 
wireless links and providing a better service than what is there.

Travis

From: NANOG  On Behalf Of Josh 
Luthman
Sent: Friday, February 11, 2022 3:15 PM
To: Brandon Svec 
Cc: NANOG 
Subject: Re: New minimum speed for US broadband connections

Because literally every case I've seen along these lines is someone complaining 
about the coax connection is "only 100 meg when I pay for 200 meg".  Comcast 
was the most hated company and yet they factually had better speeds (possibly 
in part to their subjectively terrible customer service) for years.

>An apartment building could have cheap 1G fiber and the houses across the 
>street have no option but slow DSL.

Where is this example?  Or is this strictly hypothetical?

I am not seeing any examples, anywhere, with accurate data, where it's what 
most consider to be in town/urban and poor speeds.  The only one that was close 
was Jared and I'm pretty sure when I saw the map I wouldn't consider that in 
town (could be wrong) but again, there's gig fiber there now.  I don't remember 
if he actually got his CLEC, or why that matters, but there's fiber there now.

On Fri, Feb 11, 2022 at 4:05 PM Brandon Svec via NANOG 
mailto:nanog@nanog.org>> wrote:
What is the point of these anecdotes? Surely anyone on this list with even a 
passing knowledge of the broadband landscape in the United States knows how hit 
or miss it can be.  An apartment building could have cheap 1G fiber and the 
houses across the street have no option but slow DSL.  Houses could have 
reliable high speed cable internet, but the office park across the field has no 
such choice because the buildout cost is prohibitively high to get fiber, etc.

There are plenty of places with only one or two choices of provider too.  Of 
course, this is literally changing by the minute as new services are 
continually being added and upgraded.
Brandon Svec


On Fri, Feb 11, 2022 at 12:36 PM Josh Luthman 
mailto:j...@imaginenetworksllc.com>> wrote:
OK the one example you provided has gigabit fiber though.

On Fri, Feb 11, 2022 at 8:41 AM Tom Beecher 
mailto:beec...@beecher.cc>> wrote:
Can you provide examples?

https://www.youtube.com/watch?v=Twe6uTwOyJo_channel=NANOG

Our good friend Jared could only get 1.5M DSL living just outside Ann Arbor, 
MI, so he had to start his own CLEC.

I have friends in significantly more rural areas than he lives in ( Niagara and 
Orleans county NYS , between Niagara Falls and Rochester ) who have the same 
400Mb package from Spectrum that I do, living in the City of Niagara Falls.

This is not to say that rural America is a mecca of connectivity; there is a 
long way to go all the way around regardless. But it is a direct example as you 
asked for.

On Thu, Feb 10, 2022 at 3:57 PM Josh Luthman 
mailto:j...@imaginenetworksllc.com>> wrote:
>There are plenty of urban and suburban areas in America that are far worse off 
>from a broadband perspective than “rural America”.

Can you provide examples?

On Thu, Feb 10, 2022 at 3:51 PM Owen DeLong via NANOG 
mailto:nanog@nanog.org>> wrote:


> On Jun 2, 2021, at 02:10 , Mark Tinka 
> mailto:mark@tinka.africa>> wrote:
>
>
>
> On 6/2/21 11:04, Owen DeLong wrote:
>
>> I disagree… If it could be forced into a standardized format using a 
>> standardized approach to data acquisition and reliable comparable results 
>> across providers, it could be a very useful adjunct to real competition.
>
> If we can't even agree on what "minimum speed for U.S. broadband connections" 
> actually means, fat chance having a "nutritional facts" at the back of the 
> "Internet in a tea cup" dropped off at your door step.
>
> I'm not saying it's not useful, I'm just saying that easily goes down the 
> "what color should we use for the bike shed" territory, while people in rural 
> America still have no or poor Internet access.
>
> Mark.

ROFLMAO…

People in Rural America seem to be doing just fine. Most of the ones I know at 
least have GPON or better.

Meanwhile, here in San Jose, a city that bills itself as “The Capital of 
Silicon Valley”, the best I can get is Comcast (which does finally purport to 
be Gig down), but rarely delivers that.

Yes, anything involving the federal government will get the full bike shed 
treatment no matter what we do.

There are plenty of urban and suburban areas in America that are far worse off 
from a broadband perspective than “rural America”.

Owen

Re: junos config commit question

[Nick Suan via NANOG]

I was actually interested to see if the EX series would let me do this, and it 
turns out that if STP is enabled on any of the switch interfaces, it won't: 


tevruden@core-02# delete interfaces 

{master:0}[edit]
tevruden@core-02# commit check 
[edit protocols rstp]
  'interface'
XSTP : Interface ge-0/0/0.0 is not enabled for Ethernet Switching
error: configuration check-out failed
{master:0}[edit]
tevruden@core-02# rollback 
load complete

{master:0}[edit]
 


On Fri, Feb 11, 2022, at 4:18 PM, Lyndon Nerenberg (VE7TFX/VE6BBM) wrote:
> On an EX4300 switch running JunOS 14.1 let's imagine I typed
>
>   config
>   delete interfaces
>
> before coming to my senses.  How am I supposed to back out of that
> mess?  For the life of me, after a week of reading the 3000 page
> reference manual, and endless DuckDuckGoing, I cannot see a simple
> way of just abandoning the commit.  I've got to be missing something
> stunningly obvious here because it's unthinkable that this functionality
> doesn't exist.  Help?!?
>
> The only way out I can see is to drop into the shell, make an
> uncompressed copy of juniper.conf.gz, then pop back into the config
> editor and load that over top of the editor's config view.  Surely
> there's a saner way of dealing with this.
>
> --lyndon

Re: junos config commit question

[Lyndon Nerenberg (VE7TFX/VE6BBM)]

Marco Davids via NANOG writes:
> rollback 0

OFFS 8-0  Thanks :-)

Re: junos config commit question

[Jon Lewis]

On Fri, 11 Feb 2022, Lyndon Nerenberg (VE7TFX/VE6BBM) wrote:


On an EX4300 switch running JunOS 14.1 let's imagine I typed

config
delete interfaces

before coming to my senses.  How am I supposed to back out of that
mess?  For the life of me, after a week of reading the 3000 page
reference manual, and endless DuckDuckGoing, I cannot see a simple
way of just abandoning the commit.  I've got to be missing something
stunningly obvious here because it's unthinkable that this functionality
doesn't exist.  Help?!?


What would you say if I told you a coworker once did exactly that, and did 
commit and-quit...and it had to be fixed by another coworker getting to it 
via OOB console and doing the rollback?  :)


top [not necessary in your case, if you never left top]
rollback 0
quit

Also, get into the habit of never doing a commit without first doing
top 
show | compare
so you can see what your change is actually doing to the whole config. 
i.e. if you did a show | compare at the top of the config and saw the 
entire interfaces section of the config was "removed" in the resulting 
config diff, you probably wouldn't commit.


--
 Jon Lewis, MCP :)   |  I route
 StackPath, Sr. Neteng   |  therefore you are
_ http://www.lewis.org/~jlewis/pgp for PGP public key_

Re: New minimum speed for US broadband connections

[Nathan Angelacos]

20 miles from Sacramento.

Mother-in-law has an ATT  DSLAM *at the end of her driveway*  on
the other side of the street.  ATT swears she can get internet. Until
she tries to sign up, and "oh no... wrong side of the street"

She is at 700Kbps over a WISP ... *after* she trimmed the trees to get
line of sight.

sigh.

Re: junos config commit question

[Christopher Morrow]

On Fri, Feb 11, 2022 at 5:26 PM Ryan Hamel  wrote:

> If it's before committing the changes just run "top" to get back to the
> root of the configuration tree, then "rollback 0" to go back to the version
> before any changes were made, then just "exit" out.
>
> Ryan
>
>
> On Fri, Feb 11, 2022, 2:20 PM Lyndon Nerenberg (VE7TFX/VE6BBM) <
> lyn...@orthanc.ca> wrote:
>
>> On an EX4300 switch running JunOS 14.1 let's imagine I typed
>>
>> config
>> delete interfaces
>>
>>
you may ALSO be interested in the idea that you SHOULD be doing:
  configure exclusive
  fiddle
  fart
  oops!
  exit (safe to exit, your changes will get wiped out)

note that 'configure exclusive' means other people can't ALSO change the
config out from under you (and you have locked the config, so)


> before coming to my senses.  How am I supposed to back out of that
>> mess?  For the life of me, after a week of reading the 3000 page
>> reference manual, and endless DuckDuckGoing, I cannot see a simple
>> way of just abandoning the commit.  I've got to be missing something
>> stunningly obvious here because it's unthinkable that this functionality
>> doesn't exist.  Help?!?
>>
>> The only way out I can see is to drop into the shell, make an
>> uncompressed copy of juniper.conf.gz, then pop back into the config
>> editor and load that over top of the editor's config view.  Surely
>> there's a saner way of dealing with this.
>>
>> --lyndon
>>
>

Re: junos config commit question

[Ryan Hamel]

If it's before committing the changes just run "top" to get back to the
root of the configuration tree, then "rollback 0" to go back to the version
before any changes were made, then just "exit" out.

Ryan


On Fri, Feb 11, 2022, 2:20 PM Lyndon Nerenberg (VE7TFX/VE6BBM) <
lyn...@orthanc.ca> wrote:

> On an EX4300 switch running JunOS 14.1 let's imagine I typed
>
> config
> delete interfaces
>
> before coming to my senses.  How am I supposed to back out of that
> mess?  For the life of me, after a week of reading the 3000 page
> reference manual, and endless DuckDuckGoing, I cannot see a simple
> way of just abandoning the commit.  I've got to be missing something
> stunningly obvious here because it's unthinkable that this functionality
> doesn't exist.  Help?!?
>
> The only way out I can see is to drop into the shell, make an
> uncompressed copy of juniper.conf.gz, then pop back into the config
> editor and load that over top of the editor's config view.  Surely
> there's a saner way of dealing with this.
>
> --lyndon
>

Re: junos config commit question

[Marco Davids via NANOG]

rollback 0

Op 11-02-22 om 23:18 schreef Lyndon Nerenberg (VE7TFX/VE6BBM):

On an EX4300 switch running JunOS 14.1 let's imagine I typed

config
delete interfaces

before coming to my senses.  How am I supposed to back out of that
mess?  For the life of me, after a week of reading the 3000 page
reference manual, and endless DuckDuckGoing, I cannot see a simple
way of just abandoning the commit.  I've got to be missing something
stunningly obvious here because it's unthinkable that this functionality
doesn't exist.  Help?!?

The only way out I can see is to drop into the shell, make an
uncompressed copy of juniper.conf.gz, then pop back into the config
editor and load that over top of the editor's config view.  Surely
there's a saner way of dealing with this.

--lyndon



--
Marco Davids

junos config commit question

[Lyndon Nerenberg (VE7TFX/VE6BBM)]

On an EX4300 switch running JunOS 14.1 let's imagine I typed

config
delete interfaces

before coming to my senses.  How am I supposed to back out of that
mess?  For the life of me, after a week of reading the 3000 page
reference manual, and endless DuckDuckGoing, I cannot see a simple
way of just abandoning the commit.  I've got to be missing something
stunningly obvious here because it's unthinkable that this functionality
doesn't exist.  Help?!?

The only way out I can see is to drop into the shell, make an
uncompressed copy of juniper.conf.gz, then pop back into the config
editor and load that over top of the editor's config view.  Surely
there's a saner way of dealing with this.

--lyndon

Re: New minimum speed for US broadband connections

[Blake Hudson]

The house was completed a year or two before my mother's purchase and it 
took Comcast another year or two to lay cable. Imagine buying a house 
and waiting three to four years for internet service. That does not 
qualify as "got service right away" in my mind. The frustrating part, 
for me as a bystander, was that the 10-20 year old homes in the same 
neighborhood had great service from several providers, while this group 
of 4-5 homes had only one option. Certainly opened my eyes to the fact 
that there are internet deserts in the middle of the suburbs.


Before purchasing my current home, I double checked visually that there 
were at least two internet providers in the ground and at least one of 
them was fiber before signing a contract. Turned out both were fiber 
while a coax provider was promised and did eventually deliver. I'm happy 
with my current service and its price; I attribute some of that to the 
competition in the area.


--Blake



On 2/11/2022 3:42 PM, Josh Luthman wrote:
I believe what he said was "Comcast did eventually lay cable".  That 
was in a brand new development.  It's a brand new house and got 
service right away.  What more do you want from providers?


Out in the country, yes, there are the 10k to 100k build out costs all 
the time.  But that's the country (rural).


On Fri, Feb 11, 2022 at 4:37 PM Brandon Svec via NANOG 
 wrote:


Excellent example.  I see this all.the.time. She could
probably get Comcast just fine by paying $50k buildout or signing
a 10 year agreement for TV/Phone/Internet and convincing 5
neighbors too ;)
*Brandon *



On Fri, Feb 11, 2022 at 1:32 PM Blake Hudson  wrote:

My mom moves to Olathe, KS. The realtor indicated that ATT,
Comcast, and
Google Fiber all provided service to the neighborhood and the HOA
confirmed. Unfortunately for her, Google fiber laid fiber ~3
years
before and her cul-de-sac was developed ~2 years before she
moved in. No
Google Fiber, no Comcast, just ATT. Both Comcast and Google
Fiber were
within 100 ft of her property and wouldn't serve her. Google
has no
plans to serve that cul-de-sac in the future. Comcast did
eventually lay
cable. I'm sure her and her neighbors aren't the only people
in America
to experience something similar.

On 2/11/2022 3:14 PM, Josh Luthman wrote:
>
> >An apartment building could have cheap 1G fiber and the
houses across
> the street have no option but slow DSL.
>
> Where is this example?  Or is this strictly hypothetical?
>
>

Re: New minimum speed for US broadband connections

[Josh Luthman]

I believe what he said was "Comcast did eventually lay cable".  That was in
a brand new development.  It's a brand new house and got service right
away.  What more do you want from providers?

Out in the country, yes, there are the 10k to 100k build out costs all the
time.  But that's the country (rural).

On Fri, Feb 11, 2022 at 4:37 PM Brandon Svec via NANOG 
wrote:

> Excellent example.  I see this all.the.time. She could probably get
> Comcast just fine by paying $50k buildout or signing a 10 year agreement
> for TV/Phone/Internet and convincing 5 neighbors too ;)
> *Brandon *
>
>
> On Fri, Feb 11, 2022 at 1:32 PM Blake Hudson  wrote:
>
>> My mom moves to Olathe, KS. The realtor indicated that ATT, Comcast, and
>> Google Fiber all provided service to the neighborhood and the HOA
>> confirmed. Unfortunately for her, Google fiber laid fiber ~3 years
>> before and her cul-de-sac was developed ~2 years before she moved in. No
>> Google Fiber, no Comcast, just ATT. Both Comcast and Google Fiber were
>> within 100 ft of her property and wouldn't serve her. Google has no
>> plans to serve that cul-de-sac in the future. Comcast did eventually lay
>> cable. I'm sure her and her neighbors aren't the only people in America
>> to experience something similar.
>>
>> On 2/11/2022 3:14 PM, Josh Luthman wrote:
>> >
>> > >An apartment building could have cheap 1G fiber and the houses across
>> > the street have no option but slow DSL.
>> >
>> > Where is this example?  Or is this strictly hypothetical?
>> >
>> >
>>
>>

Re: New minimum speed for US broadband connections

[Brandon Svec via NANOG]

Excellent example.  I see this all.the.time. She could probably get Comcast
just fine by paying $50k buildout or signing a 10 year agreement for
TV/Phone/Internet and convincing 5 neighbors too ;)
*Brandon *


On Fri, Feb 11, 2022 at 1:32 PM Blake Hudson  wrote:

> My mom moves to Olathe, KS. The realtor indicated that ATT, Comcast, and
> Google Fiber all provided service to the neighborhood and the HOA
> confirmed. Unfortunately for her, Google fiber laid fiber ~3 years
> before and her cul-de-sac was developed ~2 years before she moved in. No
> Google Fiber, no Comcast, just ATT. Both Comcast and Google Fiber were
> within 100 ft of her property and wouldn't serve her. Google has no
> plans to serve that cul-de-sac in the future. Comcast did eventually lay
> cable. I'm sure her and her neighbors aren't the only people in America
> to experience something similar.
>
> On 2/11/2022 3:14 PM, Josh Luthman wrote:
> >
> > >An apartment building could have cheap 1G fiber and the houses across
> > the street have no option but slow DSL.
> >
> > Where is this example?  Or is this strictly hypothetical?
> >
> >
>
>

Re: New minimum speed for US broadband connections

[Brandon Svec via NANOG]

My example is just from experience.  Not hypothetical, but also not a
specific address I can recall or feel like looking up now.

The reality on the ground as someone who sells access to smallish
businesses mostly in California is as I described.  You can't see it on a
map or database because the map may show a Comcast/att/whomever
pop/availability at an address, but to get said access across the parking
lot or street is a 6 figure build out cost and 6 months or more waiting for
permits and construction to complete so effectively a building right across
the lot or street from another has completely different options.  If you
want to zero in on an area to investigate/research I do recall fairly
recently some business parks in Hayward, CA near 880 that had no options
except bonded copper stuff up to maybe 50/50Mbps for a really high price.
One of them I sold fiber DIA to and they waited about 8 months for permits
and construction and signed a 5 year lease to reduce/avoid buildout costs.


I guess fair cost and speed are subjective, but that clarifies the point I
was making.

Best,
Brandon



On Fri, Feb 11, 2022 at 1:15 PM Josh Luthman 
wrote:

> Because literally every case I've seen along these lines is someone
> complaining about the coax connection is "only 100 meg when I pay for 200
> meg".  Comcast was the most hated company and yet they factually had better
> speeds (possibly in part to their subjectively terrible customer service)
> for years.
>
> >An apartment building could have cheap 1G fiber and the houses across the
> street have no option but slow DSL.
>
> Where is this example?  Or is this strictly hypothetical?
>
> I am not seeing any examples, anywhere, with accurate data, where it's
> what most consider to be in town/urban and poor speeds.  The only one that
> was close was Jared and I'm pretty sure when I saw the map I wouldn't
> consider that in town (could be wrong) but again, there's gig fiber there
> now.  I don't remember if he actually got his CLEC, or why that matters,
> but there's fiber there now.
>
> On Fri, Feb 11, 2022 at 4:05 PM Brandon Svec via NANOG 
> wrote:
>
>> What is the point of these anecdotes? Surely anyone on this list with
>> even a passing knowledge of the broadband landscape in the United States
>> knows how hit or miss it can be.  An apartment building could have cheap 1G
>> fiber and the houses across the street have no option but slow DSL.  Houses
>> could have reliable high speed cable internet, but the office park across
>> the field has no such choice because the buildout cost is prohibitively
>> high to get fiber, etc.
>>
>> There are plenty of places with only one or two choices of provider too.
>> Of course, this is literally changing by the minute as new services are
>> continually being added and upgraded.
>> *Brandon Svec*
>>
>>
>>
>> On Fri, Feb 11, 2022 at 12:36 PM Josh Luthman <
>> j...@imaginenetworksllc.com> wrote:
>>
>>> OK the one example you provided has gigabit fiber though.
>>>
>>> On Fri, Feb 11, 2022 at 8:41 AM Tom Beecher  wrote:
>>>
 Can you provide examples?
>

 https://www.youtube.com/watch?v=Twe6uTwOyJo_channel=NANOG

 Our good friend Jared could only get 1.5M DSL living just outside Ann
 Arbor, MI, so he had to start his own CLEC.

 I have friends in significantly more rural areas than he lives in (
 Niagara and Orleans county NYS , between Niagara Falls and Rochester ) who
 have the same 400Mb package from Spectrum that I do, living in the City of
 Niagara Falls.

 This is not to say that rural America is a mecca of connectivity; there
 is a long way to go all the way around regardless. But it is a direct
 example as you asked for.

 On Thu, Feb 10, 2022 at 3:57 PM Josh Luthman <
 j...@imaginenetworksllc.com> wrote:

> >There are plenty of urban and suburban areas in America that are far
> worse off from a broadband perspective than “rural America”.
>
> Can you provide examples?
>
> On Thu, Feb 10, 2022 at 3:51 PM Owen DeLong via NANOG 
> wrote:
>
>>
>>
>> > On Jun 2, 2021, at 02:10 , Mark Tinka  wrote:
>> >
>> >
>> >
>> > On 6/2/21 11:04, Owen DeLong wrote:
>> >
>> >> I disagree… If it could be forced into a standardized format using
>> a standardized approach to data acquisition and reliable comparable 
>> results
>> across providers, it could be a very useful adjunct to real competition.
>> >
>> > If we can't even agree on what "minimum speed for U.S. broadband
>> connections" actually means, fat chance having a "nutritional facts" at 
>> the
>> back of the "Internet in a tea cup" dropped off at your door step.
>> >
>> > I'm not saying it's not useful, I'm just saying that easily goes
>> down the "what color should we use for the bike shed" territory, while
>> people in rural America still have no or poor Internet access.
>> >

Re: New minimum speed for US broadband connections

[Blake Hudson]

My mom moves to Olathe, KS. The realtor indicated that ATT, Comcast, and 
Google Fiber all provided service to the neighborhood and the HOA 
confirmed. Unfortunately for her, Google fiber laid fiber ~3 years 
before and her cul-de-sac was developed ~2 years before she moved in. No 
Google Fiber, no Comcast, just ATT. Both Comcast and Google Fiber were 
within 100 ft of her property and wouldn't serve her. Google has no 
plans to serve that cul-de-sac in the future. Comcast did eventually lay 
cable. I'm sure her and her neighbors aren't the only people in America 
to experience something similar.


On 2/11/2022 3:14 PM, Josh Luthman wrote:


>An apartment building could have cheap 1G fiber and the houses across 
the street have no option but slow DSL.


Where is this example?  Or is this strictly hypothetical?

Re: New minimum speed for US broadband connections

[Josh Luthman]

Because literally every case I've seen along these lines is someone
complaining about the coax connection is "only 100 meg when I pay for 200
meg".  Comcast was the most hated company and yet they factually had better
speeds (possibly in part to their subjectively terrible customer service)
for years.

>An apartment building could have cheap 1G fiber and the houses across the
street have no option but slow DSL.

Where is this example?  Or is this strictly hypothetical?

I am not seeing any examples, anywhere, with accurate data, where it's what
most consider to be in town/urban and poor speeds.  The only one that was
close was Jared and I'm pretty sure when I saw the map I wouldn't consider
that in town (could be wrong) but again, there's gig fiber there now.  I
don't remember if he actually got his CLEC, or why that matters, but
there's fiber there now.

On Fri, Feb 11, 2022 at 4:05 PM Brandon Svec via NANOG 
wrote:

> What is the point of these anecdotes? Surely anyone on this list with even
> a passing knowledge of the broadband landscape in the United States knows
> how hit or miss it can be.  An apartment building could have cheap 1G fiber
> and the houses across the street have no option but slow DSL.  Houses could
> have reliable high speed cable internet, but the office park across the
> field has no such choice because the buildout cost is prohibitively high to
> get fiber, etc.
>
> There are plenty of places with only one or two choices of provider too.
> Of course, this is literally changing by the minute as new services are
> continually being added and upgraded.
> *Brandon Svec*
>
>
>
> On Fri, Feb 11, 2022 at 12:36 PM Josh Luthman 
> wrote:
>
>> OK the one example you provided has gigabit fiber though.
>>
>> On Fri, Feb 11, 2022 at 8:41 AM Tom Beecher  wrote:
>>
>>> Can you provide examples?

>>>
>>> https://www.youtube.com/watch?v=Twe6uTwOyJo_channel=NANOG
>>>
>>> Our good friend Jared could only get 1.5M DSL living just outside Ann
>>> Arbor, MI, so he had to start his own CLEC.
>>>
>>> I have friends in significantly more rural areas than he lives in (
>>> Niagara and Orleans county NYS , between Niagara Falls and Rochester ) who
>>> have the same 400Mb package from Spectrum that I do, living in the City of
>>> Niagara Falls.
>>>
>>> This is not to say that rural America is a mecca of connectivity; there
>>> is a long way to go all the way around regardless. But it is a direct
>>> example as you asked for.
>>>
>>> On Thu, Feb 10, 2022 at 3:57 PM Josh Luthman <
>>> j...@imaginenetworksllc.com> wrote:
>>>
 >There are plenty of urban and suburban areas in America that are far
 worse off from a broadband perspective than “rural America”.

 Can you provide examples?

 On Thu, Feb 10, 2022 at 3:51 PM Owen DeLong via NANOG 
 wrote:

>
>
> > On Jun 2, 2021, at 02:10 , Mark Tinka  wrote:
> >
> >
> >
> > On 6/2/21 11:04, Owen DeLong wrote:
> >
> >> I disagree… If it could be forced into a standardized format using
> a standardized approach to data acquisition and reliable comparable 
> results
> across providers, it could be a very useful adjunct to real competition.
> >
> > If we can't even agree on what "minimum speed for U.S. broadband
> connections" actually means, fat chance having a "nutritional facts" at 
> the
> back of the "Internet in a tea cup" dropped off at your door step.
> >
> > I'm not saying it's not useful, I'm just saying that easily goes
> down the "what color should we use for the bike shed" territory, while
> people in rural America still have no or poor Internet access.
> >
> > Mark.
>
> ROFLMAO…
>
> People in Rural America seem to be doing just fine. Most of the ones I
> know at least have GPON or better.
>
> Meanwhile, here in San Jose, a city that bills itself as “The Capital
> of Silicon Valley”, the best I can get is Comcast (which does finally
> purport to be Gig down), but rarely delivers that.
>
> Yes, anything involving the federal government will get the full bike
> shed treatment no matter what we do.
>
> There are plenty of urban and suburban areas in America that are far
> worse off from a broadband perspective than “rural America”.
>
> Owen
>
>

Re: New minimum speed for US broadband connections

[Brandon Svec via NANOG]

What is the point of these anecdotes? Surely anyone on this list with even
a passing knowledge of the broadband landscape in the United States knows
how hit or miss it can be.  An apartment building could have cheap 1G fiber
and the houses across the street have no option but slow DSL.  Houses could
have reliable high speed cable internet, but the office park across the
field has no such choice because the buildout cost is prohibitively high to
get fiber, etc.

There are plenty of places with only one or two choices of provider too.
Of course, this is literally changing by the minute as new services are
continually being added and upgraded.
*Brandon Svec*



On Fri, Feb 11, 2022 at 12:36 PM Josh Luthman 
wrote:

> OK the one example you provided has gigabit fiber though.
>
> On Fri, Feb 11, 2022 at 8:41 AM Tom Beecher  wrote:
>
>> Can you provide examples?
>>>
>>
>> https://www.youtube.com/watch?v=Twe6uTwOyJo_channel=NANOG
>>
>> Our good friend Jared could only get 1.5M DSL living just outside Ann
>> Arbor, MI, so he had to start his own CLEC.
>>
>> I have friends in significantly more rural areas than he lives in (
>> Niagara and Orleans county NYS , between Niagara Falls and Rochester ) who
>> have the same 400Mb package from Spectrum that I do, living in the City of
>> Niagara Falls.
>>
>> This is not to say that rural America is a mecca of connectivity; there
>> is a long way to go all the way around regardless. But it is a direct
>> example as you asked for.
>>
>> On Thu, Feb 10, 2022 at 3:57 PM Josh Luthman 
>> wrote:
>>
>>> >There are plenty of urban and suburban areas in America that are far
>>> worse off from a broadband perspective than “rural America”.
>>>
>>> Can you provide examples?
>>>
>>> On Thu, Feb 10, 2022 at 3:51 PM Owen DeLong via NANOG 
>>> wrote:
>>>


 > On Jun 2, 2021, at 02:10 , Mark Tinka  wrote:
 >
 >
 >
 > On 6/2/21 11:04, Owen DeLong wrote:
 >
 >> I disagree… If it could be forced into a standardized format using a
 standardized approach to data acquisition and reliable comparable results
 across providers, it could be a very useful adjunct to real competition.
 >
 > If we can't even agree on what "minimum speed for U.S. broadband
 connections" actually means, fat chance having a "nutritional facts" at the
 back of the "Internet in a tea cup" dropped off at your door step.
 >
 > I'm not saying it's not useful, I'm just saying that easily goes down
 the "what color should we use for the bike shed" territory, while people in
 rural America still have no or poor Internet access.
 >
 > Mark.

 ROFLMAO…

 People in Rural America seem to be doing just fine. Most of the ones I
 know at least have GPON or better.

 Meanwhile, here in San Jose, a city that bills itself as “The Capital
 of Silicon Valley”, the best I can get is Comcast (which does finally
 purport to be Gig down), but rarely delivers that.

 Yes, anything involving the federal government will get the full bike
 shed treatment no matter what we do.

 There are plenty of urban and suburban areas in America that are far
 worse off from a broadband perspective than “rural America”.

 Owen

Re: Authoritative Resources for Public DNS Pinging

[sronan]

Usage of 1.1.1.1 has been widespread amongst wireless controllers for a very 
long time, as an address for their captive portals.

Shane


> On Feb 11, 2022, at 3:44 PM, Mike Lewinski via NANOG  wrote:
> 
> On a related note, I just discovered a NID that has 1.1.1.1 assigned to the 
> outband interface by default, and it is apparently not user modifiable. So, 
> not only can these devices never use 1.1.1.1 for name resolution, but 
> attempts to determine "is the circuit up" by pinging it will always return 
> bad information. To really pour salt on the wound, this device has no 
> physical outbound interface (likely why the UI doesn't allow changing it).
> 
> Bug report filed. I don't really want to use it for either purpose, but 
> hopefully a fix saves someone else the headache. And it really chaps my  
> when public addresses get used this way.

RE: Authoritative Resources for Public DNS Pinging

[Mike Lewinski via NANOG]

On a related note, I just discovered a NID that has 1.1.1.1 assigned to the 
outband interface by default, and it is apparently not user modifiable. So, not 
only can these devices never use 1.1.1.1 for name resolution, but attempts to 
determine "is the circuit up" by pinging it will always return bad information. 
To really pour salt on the wound, this device has no physical outbound 
interface (likely why the UI doesn't allow changing it).

Bug report filed. I don't really want to use it for either purpose, but 
hopefully a fix saves someone else the headache. And it really chaps my  
when public addresses get used this way.

Re: New minimum speed for US broadband connections

[Josh Luthman]

OK the one example you provided has gigabit fiber though.

On Fri, Feb 11, 2022 at 8:41 AM Tom Beecher  wrote:

> Can you provide examples?
>>
>
> https://www.youtube.com/watch?v=Twe6uTwOyJo_channel=NANOG
>
> Our good friend Jared could only get 1.5M DSL living just outside Ann
> Arbor, MI, so he had to start his own CLEC.
>
> I have friends in significantly more rural areas than he lives in (
> Niagara and Orleans county NYS , between Niagara Falls and Rochester ) who
> have the same 400Mb package from Spectrum that I do, living in the City of
> Niagara Falls.
>
> This is not to say that rural America is a mecca of connectivity; there is
> a long way to go all the way around regardless. But it is a direct example
> as you asked for.
>
> On Thu, Feb 10, 2022 at 3:57 PM Josh Luthman 
> wrote:
>
>> >There are plenty of urban and suburban areas in America that are far
>> worse off from a broadband perspective than “rural America”.
>>
>> Can you provide examples?
>>
>> On Thu, Feb 10, 2022 at 3:51 PM Owen DeLong via NANOG 
>> wrote:
>>
>>>
>>>
>>> > On Jun 2, 2021, at 02:10 , Mark Tinka  wrote:
>>> >
>>> >
>>> >
>>> > On 6/2/21 11:04, Owen DeLong wrote:
>>> >
>>> >> I disagree… If it could be forced into a standardized format using a
>>> standardized approach to data acquisition and reliable comparable results
>>> across providers, it could be a very useful adjunct to real competition.
>>> >
>>> > If we can't even agree on what "minimum speed for U.S. broadband
>>> connections" actually means, fat chance having a "nutritional facts" at the
>>> back of the "Internet in a tea cup" dropped off at your door step.
>>> >
>>> > I'm not saying it's not useful, I'm just saying that easily goes down
>>> the "what color should we use for the bike shed" territory, while people in
>>> rural America still have no or poor Internet access.
>>> >
>>> > Mark.
>>>
>>> ROFLMAO…
>>>
>>> People in Rural America seem to be doing just fine. Most of the ones I
>>> know at least have GPON or better.
>>>
>>> Meanwhile, here in San Jose, a city that bills itself as “The Capital of
>>> Silicon Valley”, the best I can get is Comcast (which does finally purport
>>> to be Gig down), but rarely delivers that.
>>>
>>> Yes, anything involving the federal government will get the full bike
>>> shed treatment no matter what we do.
>>>
>>> There are plenty of urban and suburban areas in America that are far
>>> worse off from a broadband perspective than “rural America”.
>>>
>>> Owen
>>>
>>>

Re: VPN recommendations?

[Rich Greenwood via NANOG]

The port forwarding only applies to manual NAT traversal.  If you use auto
NAT traversal, it takes care of that.  Because all of the connections are
coordinated through the dashboard, the Auto-VPN will typically work even if
all nodes are behind NAT.  I've used them on the end of Verizon (CG-NAT)
connections and they work fine.  I have had one instance where three of
them were behind the same single IP NAT and the third would fail to
connect.  We had to get one of them moved to a different NAT IP to solve
that.

If you're looking for a simple to use, easy to manage VPN appliance, the MX
(and Z) Meraki products will work.  The config is entirely handled through
the dashboard, so no-touch, drop ship deployments are an option.  You can
provide view only access to users per network, so the customer or a first
level tech could be given the ability to look but not break anything.

All of the MX and Z products will work in a single VPN, so you can pick the
device that best fits the requirements.  For a small office with one or two
people, the Z3 works great, it even has one PoE port for an IP phone.  For
larger sites or the core site, they go up to 6Gb (I think) of throughput
for the MX450, with redundant power and uplinks.

As others have pointed out, they are license based and they don't work
without a license, and they are a Cisco product, so pricing will depend on
how good your relationship is with your Cisco rep. :)  One big caveat: they
are still lacking in the IPv6 realm so if that is a requirement, they won't
work right now.
--Rich


> -- Forwarded message --
> From: William Herrin 
> To: Shawn L 
> Cc: "nanog@nanog.org" 
> Bcc:
> Date: Thu, 10 Feb 2022 10:54:39 -0800
> Subject: Re: VPN recommendations?
> On Thu, Feb 10, 2022 at 10:18 AM Shawn L  wrote:
> > Meraki MX series? Dynamic IPs and NATs don't really cause them a
> problem.  Some CGNats do (AT I'm looking at you).
>
> Thanks Shawn,
>
> The documentation I found at
>
> https://documentation.meraki.com/MX/Site-to-site_VPN/Site-to-Site_VPN_Settings
> suggests that the NAT firewall has to be explicitly configured to
> deliver UDP 500/4500 to the Meraki behind it. Are you aware of any
> documentation that describes:
>
> LAN - Meraki - NAT (dynaimic IP) - Internet - (static IP) Meraki - LAN
>
> Where the left-side Meraki is responsible for establishing and keeping
> the NAT translations alive without any special configuration on the
> NAT?
>
> Regards,
> Bill
>


-- 
Rich Greenwood
Network Engineer
Shasta County Office of Education

Information Technology

1644 Magnolia Ave.

Redding, CA 96001

Office: 530-225-0161

Hotline: 530-225-0279

rgreenw...@shastacoe.org


smime.p7s
Description: S/MIME Cryptographic Signature

Re: VPN recommendations?

[William Herrin]

On Fri, Feb 11, 2022 at 10:35 AM Dan Sneddon  wrote:
> 1) IPSEC does not lend itself to dynamic routing or dynamic configuration. It 
> is very much a static set-it-and-forget-it technology, but that doesn’t work 
> in a dynamically changing environment.

Hi Dan,

Depending on how you configure it, IPSEC can work fine with dynamic
routing. The thing to understand is that IPSec has two modes:
transport and tunnel. Transport is between exactly two IP addresses
while tunnel expects a broader network to exist on at least one end.
"Tunnel" mode is what everyone actually uses but you can deconstruct
it: it's built up from transport mode + a tunnel protocol (gre or ipip
I don't remember which) + implicit routing and firewalling which
wreaks havoc on dynamic routing. Now, it turns out that you can
instead configure IPSec in transport mode, configure the tunnel
separately and leave out the implicit firewalling.

> This may not apply to William Herrin’s (OP) use case of a VPN appliance

It's not relevant to my situation, no. I need the VPN to establish a
statically addressed clean layer 3 on top of dynamically addressed and
natted endpoints to support the next appliance in the chain where
dynamic addressing is not possible. I don't actually care if it adds
security; it just needs to establish that statically addressed layer.
Oh yeah, and it has to be listed under "virtual private network" on
the government NIAP list.
https://www.niap-ccevs.org/product/PCL.cfm?ID624=34

Regards,
Bill Herrin

-- 
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: VPN recommendations?

[Mel Beckman]

Dan,

One point you didn’t touch on is that IPSec is integrated into IPv6, typically 
hardware-accelerated on the NIC, enabling device-to-device VPNs, mitigates most 
of the dynamic issues associated with network-to-network IPSec over IPv4.

Yes, I realize IPv4 is hanging around longer than most expect, but in some 
cases I think you can make a case for deploying IPv6 just on the VPN benefits 
alone. With no public-facing services, IPv6 is already deployed in most LANs as 
a direct result of its use by modern OSes for inter-LAN communication. All you 
typically need to do is enable IPv6 at the gateway.

 -mel


On Feb 11, 2022, at 10:33 AM, Dan Sneddon 
mailto:sned...@gmail.com>> wrote:

Thank you Joy for de-lurking. I actually was not familiar with ZeroTier, and 
this is a space that I thought I was quite familiar with, so I’m glad you 
brought it to everyone’s attention. I will look further at ZeroTier, it looks 
very interesting.

I am also a very long-time lurker (although I was a NANOG list admin ~10 years 
ago) who is emerging to join this conversation.

I have recently been doing some work to evaluate and develop VPN solutions for 
connecting multiple data center cloud environments, including low-power small 
edge sites, and I have some thoughts about the current state of the art to 
share.

Until recently a very strong proponent of IPSEC. I liked the way IPSEC was 
placed within the OSI model directly at layer 3, unlike some of the VPN 
technologies which operate above or below layer 3. However I do not believe 
that IPSEC is future-proof, for the following two reasons:

1) IPSEC does not lend itself to dynamic routing or dynamic configuration. It 
is very much a static set-it-and-forget-it technology, but that doesn’t work in 
a dynamically changing environment.

2) IPSEC does not always lend itself to hardware offloading in the way some 
other technologies do. Some NICs do support hardware acceleration for IPSEC, 
but this does not always integrate well with kernel or user space when you are 
integrating virtual network functions (VNFs) like 
routers/firewalls/load-balancers.

Wireguard works well in dynamic environments. TLS using something like OpenSSL 
does as well. Both provide key advantages, particularly on top of Linux.

* Support for hardware offloads such as TCP segmentation provide vast 
improvements in performance on higher-end x86 hardware. Some recent testing I 
have been shown proves that TCP segmentation offload can provide more than a 5X 
speedup compared to other HW offloads without TCP segmentation (from 5Gb/s to 
above 25Gb/s in some tests).

* With the right encryption algorithm CPU acceleration for cryptography reduces 
CPU load and increases performance.

* Integration with kernel routing provides the ability to integrate with 
dynamic routing such as BGP daemons (e.g. FRRouting, etc.).

* In recent Linux kernels eBPF/XDP provide a hardware interface to the kernel 
which accelerates network throughput to near line-rate, while minimizing CPU 
impact.

This may not apply to William Herrin’s (OP) use case of a VPN appliance for 
100mbps to 1gbps speeds, but it is something to keep in mind for building 
higher performance solutions or for planning for increasing bandwidth in the 
future. For the 100mbps+ use case I have had success building appliances using 
OpenVPN on top of certain ARM based platforms like Marvell Armada, or 
single-board computers with Intel CPUs with AES-NI acceleration. I am currently 
looking at implementing Wireguard on the same platforms. For a simple low-power 
ARM router appliance the Turris Omnia has been a great fully open platform 
running a custom LEDE/OpenWRT OS. The Turris Mox provides a modular hardware 
platform for expandability, albeit with slightly less performance. Both of 
these platforms are developed by the engineers at CZ.nic, the TLD registrar for 
the Czech Republic.

https://secure.nic.cz/files/Turris-web/Omnia/Omnia2020_datasheet.pdf

https://www.turris.com/en/mox/overview/

-Dan Sneddon

On Feb 10, 2022, at 10:51 AM, j...@cleverhack.com 
wrote:

Hello NANOG,

My name is Joy Larkin and I'm actually a long-time years-long lurker on the 
NANOG list (I have v odd hobbies) and I am also ZeroTier's Head of Marketing. I 
know I'm not supposed to be too promotional on here, but I'd love to see some 
of you pick up ZT.

Our founder, Adam Ierymenko just did a talk at Networking Field Day 27, here 
are two of the recordings from that session:

* ZeroTier The Planetary Data Center
   * https://www.youtube.com/watch?v=T2BbrqpnMAE

* ZeroTier Technical Deep Dive
   * https://www.youtube.com/watch?v=VhQ30bVF3_s

If you have questions, let me know - you can reach me at 
joy.lar...@zerotier.com

Best,
-Joy

On 2022-02-10 10:12, Mike Lyon wrote:
How about running ZeroTier on those Linux boxes and call it a day?
https://www.zerotier.com/
-Mike
On Feb 10, 2022, at 10:07, David Guo via NANOG 

Re: VPN recommendations?

[Dan Sneddon]

Thank you Joy for de-lurking. I actually was not familiar with ZeroTier, and 
this is a space that I thought I was quite familiar with, so I’m glad you 
brought it to everyone’s attention. I will look further at ZeroTier, it looks 
very interesting.

I am also a very long-time lurker (although I was a NANOG list admin ~10 years 
ago) who is emerging to join this conversation.

I have recently been doing some work to evaluate and develop VPN solutions for 
connecting multiple data center cloud environments, including low-power small 
edge sites, and I have some thoughts about the current state of the art to 
share.

Until recently a very strong proponent of IPSEC. I liked the way IPSEC was 
placed within the OSI model directly at layer 3, unlike some of the VPN 
technologies which operate above or below layer 3. However I do not believe 
that IPSEC is future-proof, for the following two reasons:

1) IPSEC does not lend itself to dynamic routing or dynamic configuration. It 
is very much a static set-it-and-forget-it technology, but that doesn’t work in 
a dynamically changing environment.

2) IPSEC does not always lend itself to hardware offloading in the way some 
other technologies do. Some NICs do support hardware acceleration for IPSEC, 
but this does not always integrate well with kernel or user space when you are 
integrating virtual network functions (VNFs) like 
routers/firewalls/load-balancers.

Wireguard works well in dynamic environments. TLS using something like OpenSSL 
does as well. Both provide key advantages, particularly on top of Linux.

* Support for hardware offloads such as TCP segmentation provide vast 
improvements in performance on higher-end x86 hardware. Some recent testing I 
have been shown proves that TCP segmentation offload can provide more than a 5X 
speedup compared to other HW offloads without TCP segmentation (from 5Gb/s to 
above 25Gb/s in some tests).

* With the right encryption algorithm CPU acceleration for cryptography reduces 
CPU load and increases performance.

* Integration with kernel routing provides the ability to integrate with 
dynamic routing such as BGP daemons (e.g. FRRouting, etc.).

* In recent Linux kernels eBPF/XDP provide a hardware interface to the kernel 
which accelerates network throughput to near line-rate, while minimizing CPU 
impact.

This may not apply to William Herrin’s (OP) use case of a VPN appliance for 
100mbps to 1gbps speeds, but it is something to keep in mind for building 
higher performance solutions or for planning for increasing bandwidth in the 
future. For the 100mbps+ use case I have had success building appliances using 
OpenVPN on top of certain ARM based platforms like Marvell Armada, or 
single-board computers with Intel CPUs with AES-NI acceleration. I am currently 
looking at implementing Wireguard on the same platforms. For a simple low-power 
ARM router appliance the Turris Omnia has been a great fully open platform 
running a custom LEDE/OpenWRT OS. The Turris Mox provides a modular hardware 
platform for expandability, albeit with slightly less performance. Both of 
these platforms are developed by the engineers at CZ.nic, the TLD registrar for 
the Czech Republic.

https://secure.nic.cz/files/Turris-web/Omnia/Omnia2020_datasheet.pdf

https://www.turris.com/en/mox/overview/

-Dan Sneddon

> On Feb 10, 2022, at 10:51 AM, j...@cleverhack.com wrote:
> 
> Hello NANOG,
> 
> My name is Joy Larkin and I'm actually a long-time years-long lurker on the 
> NANOG list (I have v odd hobbies) and I am also ZeroTier's Head of Marketing. 
> I know I'm not supposed to be too promotional on here, but I'd love to see 
> some of you pick up ZT.
> 
> Our founder, Adam Ierymenko just did a talk at Networking Field Day 27, here 
> are two of the recordings from that session:
> 
> * ZeroTier The Planetary Data Center
>* https://www.youtube.com/watch?v=T2BbrqpnMAE
> 
> * ZeroTier Technical Deep Dive
>* https://www.youtube.com/watch?v=VhQ30bVF3_s
> 
> If you have questions, let me know - you can reach me at 
> joy.lar...@zerotier.com
> 
> Best,
> -Joy
> 
>> On 2022-02-10 10:12, Mike Lyon wrote:
>> How about running ZeroTier on those Linux boxes and call it a day?
>> https://www.zerotier.com/
>> -Mike
>>> On Feb 10, 2022, at 10:07, David Guo via NANOG 
>>> wrote:
>>> 
>>> You may try WireGuard and use ddns
>>> From: NANOG  On Behalf Of
>>> William Herrin
>>> Sent: Friday, February 11, 2022 2:02 AM
>>> To: nanog@nanog.org
>>> Subject: VPN recommendations?
>>> Hi folks,
>>> Do you have any recommendations for VPN appliances? Specifically: I
>>> need to build a site to site VPNs at speeds between 100mpbs and 1
>>> gbit where all but one of the sites are behind an IPv4 NAT gateway
>>> with dynamic public IP addresses.
>>> Normally I'd throw OpenVPN on a couple of Linux boxes and be happy
>>> but my customer insists on a network appliance. Site to site VPNs
>>> using IPSec and static IP addresses on the plaintext side are a dime
>>> a 

Cryptocurrency attack due to BGP hijacking

[Andrew Wesie]

Recently, there was an attack on Klayswap [1] believed to be due to
BGP hijacking [2]. From the public data on routeviews, we can see that
there were announcements for the hijacked IP ranges, for example:

U|A|1643854199.00|routeviews|route-views.wide|||2497|202.249.2.169|211.249.221.0/24|202.249.2.169|2497
6461 9457|9457|||

The weird part is that the path from AS6461 to AS9457 does not show up
in any other routes. As far as I can tell from public information,
there is no transit nor peering relationship between AS6461 and
AS9457. As such, it seems likely a peer or customer of AS6461 was
impersonating AS9457.

I sent an email to Zayo's abuse email asking if they could provide any
additional information but did not receive a response. If anyone has
additional information, please reach out. Especially information about
where the announcement may have originated.

--
Andrew Wesie
Theori, Inc.

[1] https://medium.com/klayswap/klayswap-compensation-plan-99ec4db6742f
[2] 
https://economist.co.kr/2022/02/07/column/expertColumn/20220207070013230.html

Weekly Global IPv4 Routing Table Report

[Routing Table Analysis Role Account]

This is an automated weekly mailing describing the state of the Global
IPv4 Routing Table as seen from APNIC's router in Japan.

The posting is sent to APOPS, NANOG, AfNOG, SANOG, PacNOG, SAFNOG
TZNOG, MENOG, BJNOG, SDNOG, CMNOG, LACNOG and the RIPE Routing WG.

Daily listings are sent to bgp-st...@lists.apnic.net.

For historical data, please see https://thyme.apnic.net.

If you have any comments please contact Philip Smith .

IPv4 Routing Table Report   04:00 +10GMT Sat 12 Feb, 2022

  BGP Table (Global) as seen in Japan.

Report Website: https://thyme.apnic.net
Detailed Analysis:  https://thyme.apnic.net/current/

Analysis Summary


BGP routing table entries examined:  888216
Prefixes after maximum aggregation (per Origin AS):  334710
Deaggregation factor:  2.65
Unique aggregates announced (without unneeded subnets):  427455
Total ASes present in the Internet Routing Table: 72833
Prefixes per ASN: 12.20
Origin-only ASes present in the Internet Routing Table:   62489
Origin ASes announcing only one prefix:   25802
Transit ASes present in the Internet Routing Table:   10344
Transit-only ASes present in the Internet Routing Table:368
Average AS path length visible in the Internet Routing Table:   4.3
Max AS path length visible:  53
Max AS path prepend of ASN (265020)  50
Prefixes from unregistered ASNs in the Routing Table:   962
Number of instances of unregistered ASNs:   965
Number of 32-bit ASNs allocated by the RIRs:  38545
Number of 32-bit ASNs visible in the Routing Table:   32109
Prefixes from 32-bit ASNs in the Routing Table:  149645
Number of bogon 32-bit ASNs visible in the Routing Table:22
Special use prefixes present in the Routing Table:1
Prefixes being announced from unallocated address space:562
Number of addresses announced to Internet:   3068928512
Equivalent to 182 /8s, 236 /16s and 34 /24s
Percentage of available address space announced:   82.9
Percentage of allocated address space announced:   82.9
Percentage of available address space allocated:  100.0
Percentage of address space in use by end-sites:   99.5
Total number of prefixes smaller than registry allocations:  300423

APNIC Region Analysis Summary
-

Prefixes being announced by APNIC Region ASes:   233450
Total APNIC prefixes after maximum aggregation:   65864
APNIC Deaggregation factor:3.54
Prefixes being announced from the APNIC address blocks:  228289
Unique aggregates announced from the APNIC address blocks:94179
APNIC Region origin ASes present in the Internet Routing Table:   12370
APNIC Prefixes per ASN:   18.46
APNIC Region origin ASes announcing only one prefix:   3541
APNIC Region transit ASes present in the Internet Routing Table:   1714
Average APNIC Region AS path length visible:4.6
Max APNIC Region AS path length visible: 29
Number of APNIC region 32-bit ASNs visible in the Routing Table:   7566
Number of APNIC addresses announced to Internet:  774475904
Equivalent to 46 /8s, 41 /16s and 144 /24s
APNIC AS Blocks4608-4864, 7467-7722, 9216-10239, 17408-18431
(pre-ERX allocations)  23552-24575, 37888-38911, 45056-46079, 55296-56319,
   58368-59391, 63488-64098, 64297-64395, 131072-151865
APNIC Address Blocks 1/8,  14/8,  27/8,  36/8,  39/8,  42/8,  43/8,
49/8,  58/8,  59/8,  60/8,  61/8, 101/8, 103/8,
   106/8, 110/8, 111/8, 112/8, 113/8, 114/8, 115/8,
   116/8, 117/8, 118/8, 119/8, 120/8, 121/8, 122/8,
   123/8, 124/8, 125/8, 126/8, 133/8, 150/8, 153/8,
   163/8, 171/8, 175/8, 180/8, 182/8, 183/8, 202/8,
   203/8, 210/8, 211/8, 218/8, 219/8, 220/8, 221/8,
   222/8, 223/8,

ARIN Region Analysis Summary


Prefixes being announced by ARIN Region ASes:258538
Total ARIN prefixes after maximum aggregation:   119014
ARIN Deaggregation factor: 2.17
Prefixes being announced from the ARIN address blocks:   258644
Unique aggregates announced from the ARIN address blocks:123403
ARIN Region origin ASes present in the Internet Routing Table:18983
ARIN Prefixes per ASN:   

Re: VPN recommendations?

[Mike Hammett]

Mikrotik with RouterOS v7 with WireGuard or ZeroTier were the first things I 
thought of, but it might be a a bit premature for a production environment. In 
a year, I'd have no problem recommending that. 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

- Original Message -

From: "Ander Punnar"  
Cc: nanog@nanog.org 
Sent: Thursday, February 10, 2022 2:04:57 PM 
Subject: Re: VPN recommendations? 

On Thu, 10 Feb 2022 10:55:40 -0800, William Herrin wrote: 
> My understanding is that Wireguard is software available for general 
> purpose operating systems. I specifically need a set of hardware 
> network appliances. 

MikroTik (hardware) RouterOS (software) version 7 has WireGuard: 

https://help.mikrotik.com/docs/display/ROS/WireGuard 

Re: Authoritative Resources for Public DNS Pinging

[Grant Taylor via NANOG]

On 2/11/22 7:58 AM, Jon Lewis wrote:
8.8.8.8 is already anycasted.  What if each large ISP (for whatever 
definition of large floats your boat) setup their own internal 
instance(s) of 8.8.8.8 with a caching DNS server listening, and handled 
the traffic without bothering GOOG?


I've pontificated doing this.  On one hand I think it's a neat technical 
solution.  On the other hand I think how ... displeased I would be if 
someone were to anycast one of my services without my knowledge, much 
less consent for them to do so.  Thus I've never done it where I had a 
choice.


I believe that anycasting resources from another organization /without/ 
their consent is a hard fail and non-starter.  Independent of how pure 
the intentions are.


For users using 8.8.8.8 as a lighthouse, this would change the meaning 
of their test...i.e. a response means their connection to their ISP is 
up, and the ISP's network works at least enough to reach an internal 
8.8.8.8, but the question of their connectivity to the rest of the 
Internet would be unanswered.


I say "where I had a choice" because I have anycasted 8.8.8.8 (for ICMP 
and DNS) in an offline lab ~> D.R. exercise environment /explicitly/ 
because other systems therein had been configured to test reach ability 
to 8.8.8.8 et al.  Thus my hand was forced /inside/ the D.R. environment.




--
Grant. . . .
unix || die



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Authoritative Resources for Public DNS Pinging

[Kord Martin]

On 2022-02-11 10:11 a.m., Mike Hammett wrote:
A system always checking to see if "Internet" is up is different than 
"I think something is wrong, let me check".


Yeah. I've had ping tests fail in false-positive and false-negative 
scenarios and the take away isn't that there IS a problem, but rather an 
investigation should probably take place. I don't think anybody here is 
trying to argue that pings (or DNS lookups) are an infallible 
reachability index for "the internet".



When it comes to customers, ping tests are a non-issue because the 
complaint is normally that YouTube, Facebook, or whatever service isn't 
available and therefore the "internet is down". At some point you have 
to weight the cost/benefit of explaining to customers that the internet 
is a large collection of interconnected networks and not some "cloud" 
that we tap into. It is a series of tubes after all.



K

Re: Authoritative Resources for Public DNS Pinging

[J. Hellenthal via NANOG]

Huh


-- 

J. Hellenthal

The fact that there's a highway to Hell but only a stairway to Heaven says a 
lot about anticipated traffic volume.






> On Feb 11, 2022, at 09:10, Tom Beecher  wrote:
> 
> I am disappointed but not surprised to see this discussion on NANOG. 
> Encouraging Users to use a tool (that is often ignored by the hardware 
> targeted) by providing a non-revenue-creating special target does not make 
> business sense.
> 
> To be fair, I don't think this is unique to this community. Plenty of 
> conversations on the IETF lists that are fundamentally the same. ( Proposals 
> to change X or implement standard Y to solve something that is already 
> solvable with current tech and standards. ) Really it's just the complexity 
> of the existing solution that's different. :) 
> 
> On Fri, Feb 11, 2022 at 9:51 AM james.cut...@consultant.com 
>  wrote:
> On Feb 11, 2022, at 8:33 AM, Tom Beecher  wrote:
>> 
>> The prediciate assumption that "pinging one destination is a valid check 
>> that my internet works' is INCORRECT. There is no magical unicorn that could 
>> be built that could make that true, and 'they're gonna do it anyways' is a 
>> poor excuse to even consider it. 
>> 
> 
> The predicate assumption that unsuccessful pinging one destination is a valid 
> check that my internet DOES NOT work' is  ALSO INCORRECT. Still no magical 
> unicorn. 
> 
> I am disappointed but not surprised to see this discussion on NANOG. 
> Encouraging Users to use a tool (that is often ignored by the hardware 
> targeted) by providing a non-revenue-creating special target does not make 
> business sense.
> 
> An allied issue is educating ‘Users’ about traceroute AKA sequential ping 
> with TTL progression:
> 
>   •  Seeing missing or excessively long traceroute results from 
> intermediate nodes does NOT indicate a real problem, especially when the 
> target node is reachable with acceptable delay. 
> 
> I’ve lost count of my replies on user forums explaining this issue, even to 
> otherwise well educated users. 
> 
> To be blunt, browsing to amazon.com, apple.com or another vendor site is a 
> simple and easy to teach Internet aliveness check and, at least, offers the 
> chance for the targeted vendor site to receive revenue from sales. I have no 
> crisis of conscience from clicking an vendor shortcut for a basic end-to-end 
> Internet functional test. Or for teaching a User to do the same. This meets 
> the business purpose locally and requires no $pecial effort from Users, 
> network providers, or target systems. This precludes memorization of IP 
> addresses by end Users thus reducing the offered load from the likes of 
> excessive ping 8.8.8.8. 
> 
> I would expect NANOG members to have favorite ping target addresses based on 
> their environment, e.g., default router and a few designated targets. These 
> are useful for manual debugging but, as mentioned previously, are not 
> suitable as singular input to network monitoring.

Re: Authoritative Resources for Public DNS Pinging

[Mike Hammett]

I think we need to deliniate the conversation for human-memorable, on-demand 
needs vs. always-on configured needs. 




A system always checking to see if "Internet" is up is different than "I think 
something is wrong, let me check". 


For the always-on systems, how extensive do you want to get? What is your 
action if it's up? What is your action if it's down? 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

- Original Message -

From: "Mike Hammett"  
To: nanog@nanog.org 
Sent: Tuesday, February 8, 2022 11:56:44 AM 
Subject: Authoritative Resources for Public DNS Pinging 


Yes, pinging public DNS servers is bad. 


Googling didn't help me find anything. 


Are there any authoritative resources from said organizations saying you 
shouldn't use their servers for your persistent ping destinations? 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

Re: Authoritative Resources for Public DNS Pinging

[Tom Beecher]

>
> I am disappointed but not surprised to see this discussion on NANOG.
> Encouraging Users to use a tool (that is often ignored by the hardware
> targeted) by providing a non-revenue-creating special target does not make
> business sense.
>

To be fair, I don't think this is unique to this community. Plenty of
conversations on the IETF lists that are fundamentally the same. (
Proposals to change X or implement standard Y to solve something that is
already solvable with current tech and standards. ) Really it's just the
complexity of the existing solution that's different. :)

On Fri, Feb 11, 2022 at 9:51 AM james.cut...@consultant.com <
james.cut...@consultant.com> wrote:

> On Feb 11, 2022, at 8:33 AM, Tom Beecher  wrote:
>
>
> The prediciate assumption that "pinging one destination is a valid check
> that my internet works' is INCORRECT. There is no magical unicorn that
> could be built that could make that true, and 'they're gonna do it anyways'
> is a poor excuse to even consider it.
>
>
> The predicate assumption that unsuccessful pinging one destination is a
> valid check that my internet DOES NOT work' is  ALSO INCORRECT. Still no
> magical unicorn.
>
> I am disappointed but not surprised to see this discussion on NANOG.
> Encouraging Users to use a tool (that is often ignored by the hardware
> targeted) by providing a non-revenue-creating special target does not make
> business sense.
>
> An allied issue is educating ‘Users’ about traceroute AKA sequential ping
> with TTL progression:
>
>
>-  Seeing missing or excessively long traceroute results from
>intermediate nodes does NOT indicate a real problem, especially when the
>target node is reachable with acceptable delay.
>
>
> I’ve lost count of my replies on user forums explaining this issue, even
> to otherwise well educated users.
>
> To be blunt, browsing to amazon.com, apple.com or another vendor site is
> a simple and easy to teach Internet aliveness check and, at least, offers
> the chance for the targeted vendor site to receive revenue from sales. I
> have no crisis of conscience from clicking an vendor shortcut for a basic
> end-to-end Internet functional test. Or for teaching a User to do the same.
> This meets the business purpose locally and requires no $pecial effort from
> Users, network providers, or target systems. This precludes memorization of
> IP addresses by end Users thus reducing the offered load from the likes of
> excessive ping 8.8.8.8.
>
> I would expect NANOG members to have favorite ping target addresses based
> on their environment, e.g., default router and a few designated targets.
> These are useful for manual debugging but, as mentioned previously, are not
> suitable as singular input to network monitoring.
>

Re: Authoritative Resources for Public DNS Pinging

[Joe Greco]

On Fri, Feb 11, 2022 at 09:58:19AM -0500, Jon Lewis wrote:
> So...here's a pair of "what if"s:
> 
> What if instead of pinging 8.8.8.8, all these things using it to "test the 
> Internet" sent it DNS requests instead?  i.e.
> GOOG=$(dig +short @8.8.8.8 google.com)
> if [ -z "$GOOG" ] ; then
>   echo FAIL
> fi 
> Would that make things better or worse for GOOG (Trading lots more DNS 
> requests for the ICMP echo requests)?


ping is relatively ubiquitous.  There are certainly platforms on which
it isn't installed, but compare/contrast to the DNS options.  Is it
host?  nslookup?  dig?  No tool?

"ping internet" or "ping 8.8.8.8" are fairly straightforward by
comparison. 

> 8.8.8.8 is already anycasted.  What if each large ISP (for whatever 
> definition of large floats your boat) setup their own internal instance(s) 
> of 8.8.8.8 with a caching DNS server listening, and handled the traffic 
> without bothering GOOG?  For users using 8.8.8.8 as a lighthouse, this 
> would change the meaning of their test...i.e. a response means their 
> connection to their ISP is up, and the ISP's network works at least enough 
> to reach an internal 8.8.8.8, but the question of their connectivity to 
> the rest of the Internet would be unanswered.

Certainly that is true.  Hijacking of any mechanism is a potential risk.
Tying it into the DNS somehow at least introduces the opportunity for
DNSSEC to reduce the chance of an ISP to muck with the intended result.

We could even call it the Enhanced Link Verification Internet Service.

"ping elvis"  :-P

... JG
-- 
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"The strain of anti-intellectualism has been a constant thread winding its way
through our political and cultural life, nurtured by the false notion that
democracy means that 'my ignorance is just as good as your knowledge.'"-Asimov

Re: Authoritative Resources for Public DNS Pinging

[Jon Lewis]

On Fri, 11 Feb 2022, Mark Tinka wrote:


100% - and this is the crux of the issue.

As a community, it is clear that there is a need for this, and if 8.8.8.8 
stops being an anchor for liveliness detection, users will find something 
else to replace it with. And we can bet all our Kwacha that it won't have 
been designed for that purpose, either.


I have to admit, I haven't read most of this thread, but I am well aware 
of the issues with both end users and "routers" / firewalls pinging 
8.8.8.8 as a means of verifying "that path to the Internet is working".  I 
know GOOG doesn't appreciate the amount of ICMP echo requests their 
8.8.8.8 instances receive, and that at various times/places, that ICMP 
traffic is/has been policed by GOOG.


So...here's a pair of "what if"s:

What if instead of pinging 8.8.8.8, all these things using it to "test the 
Internet" sent it DNS requests instead?  i.e.

GOOG=$(dig +short @8.8.8.8 google.com)
if [ -z "$GOOG" ] ; then
  echo FAIL
fi 
Would that make things better or worse for GOOG (Trading lots more DNS 
requests for the ICMP echo requests)?


8.8.8.8 is already anycasted.  What if each large ISP (for whatever 
definition of large floats your boat) setup their own internal instance(s) 
of 8.8.8.8 with a caching DNS server listening, and handled the traffic 
without bothering GOOG?  For users using 8.8.8.8 as a lighthouse, this 
would change the meaning of their test...i.e. a response means their 
connection to their ISP is up, and the ISP's network works at least enough 
to reach an internal 8.8.8.8, but the question of their connectivity to 
the rest of the Internet would be unanswered.


--
 Jon Lewis, MCP :)   |  I route
 StackPath, Sr. Neteng   |  therefore you are
_ http://www.lewis.org/~jlewis/pgp for PGP public key_

Re: Authoritative Resources for Public DNS Pinging

[james.cut...@consultant.com]

On Feb 11, 2022, at 8:33 AM, Tom Beecher  wrote:
> 
> The prediciate assumption that "pinging one destination is a valid check that 
> my internet works' is INCORRECT. There is no magical unicorn that could be 
> built that could make that true, and 'they're gonna do it anyways' is a poor 
> excuse to even consider it. 
> 

The predicate assumption that unsuccessful pinging one destination is a valid 
check that my internet DOES NOT work' is  ALSO INCORRECT. Still no magical 
unicorn. 

I am disappointed but not surprised to see this discussion on NANOG. 
Encouraging Users to use a tool (that is often ignored by the hardware 
targeted) by providing a non-revenue-creating special target does not make 
business sense.

An allied issue is educating ‘Users’ about traceroute AKA sequential ping with 
TTL progression:

 Seeing missing or excessively long traceroute results from intermediate nodes 
does NOT indicate a real problem, especially when the target node is reachable 
with acceptable delay. 

I’ve lost count of my replies on user forums explaining this issue, even to 
otherwise well educated users. 

To be blunt, browsing to amazon.com, apple.com or another vendor site is a 
simple and easy to teach Internet aliveness check and, at least, offers the 
chance for the targeted vendor site to receive revenue from sales. I have no 
crisis of conscience from clicking an vendor shortcut for a basic end-to-end 
Internet functional test. Or for teaching a User to do the same. This meets the 
business purpose locally and requires no $pecial effort from Users, network 
providers, or target systems. This precludes memorization of IP addresses by 
end Users thus reducing the offered load from the likes of excessive ping 
8.8.8.8. 

I would expect NANOG members to have favorite ping target addresses based on 
their environment, e.g., default router and a few designated targets. These are 
useful for manual debugging but, as mentioned previously, are not suitable as 
singular input to network monitoring.

Re: New minimum speed for US broadband connections

[Tom Beecher]

>
> Can you provide examples?
>

https://www.youtube.com/watch?v=Twe6uTwOyJo_channel=NANOG

Our good friend Jared could only get 1.5M DSL living just outside Ann
Arbor, MI, so he had to start his own CLEC.

I have friends in significantly more rural areas than he lives in ( Niagara
and Orleans county NYS , between Niagara Falls and Rochester ) who have the
same 400Mb package from Spectrum that I do, living in the City of Niagara
Falls.

This is not to say that rural America is a mecca of connectivity; there is
a long way to go all the way around regardless. But it is a direct example
as you asked for.

On Thu, Feb 10, 2022 at 3:57 PM Josh Luthman 
wrote:

> >There are plenty of urban and suburban areas in America that are far
> worse off from a broadband perspective than “rural America”.
>
> Can you provide examples?
>
> On Thu, Feb 10, 2022 at 3:51 PM Owen DeLong via NANOG 
> wrote:
>
>>
>>
>> > On Jun 2, 2021, at 02:10 , Mark Tinka  wrote:
>> >
>> >
>> >
>> > On 6/2/21 11:04, Owen DeLong wrote:
>> >
>> >> I disagree… If it could be forced into a standardized format using a
>> standardized approach to data acquisition and reliable comparable results
>> across providers, it could be a very useful adjunct to real competition.
>> >
>> > If we can't even agree on what "minimum speed for U.S. broadband
>> connections" actually means, fat chance having a "nutritional facts" at the
>> back of the "Internet in a tea cup" dropped off at your door step.
>> >
>> > I'm not saying it's not useful, I'm just saying that easily goes down
>> the "what color should we use for the bike shed" territory, while people in
>> rural America still have no or poor Internet access.
>> >
>> > Mark.
>>
>> ROFLMAO…
>>
>> People in Rural America seem to be doing just fine. Most of the ones I
>> know at least have GPON or better.
>>
>> Meanwhile, here in San Jose, a city that bills itself as “The Capital of
>> Silicon Valley”, the best I can get is Comcast (which does finally purport
>> to be Gig down), but rarely delivers that.
>>
>> Yes, anything involving the federal government will get the full bike
>> shed treatment no matter what we do.
>>
>> There are plenty of urban and suburban areas in America that are far
>> worse off from a broadband perspective than “rural America”.
>>
>> Owen
>>
>>

Re: Authoritative Resources for Public DNS Pinging

[Tom Beecher]

>
> As a community, it is clear that there is a need for this, and if
> 8.8.8.8 stops being an anchor for liveliness detection, users will find
> something else to replace it with. And we can bet all our Kwacha that it
> won't have been designed for that purpose, either.
>

I respectfully strongly disagree on 'need'.

Let's perform a thought experiment. Assert that 8.8.8.8 was expressly
codified by Google to be a designated ICMP endpoint, and that for 100% of
ICMP echo requests they receive, they guarantee an echo-reply will be sent.
There are countless reasons , even with that (unreasonable) assumption of
100% uptime of the endpoint, that echo-requests may not reach them, or that
echo-replies may be sent but not reach the originating source. Extend this
idea even further. Assert that it is now not just Google running it, and
the largest networks in the world all agree to anycast it from their
networks.Assert is still guaranteed to respond to 100% if all echo requests
it receives, wherever it receives. ( An even more unreasonable assertion
than before!) There are STILL countless reasons why an endpoint may, at
times, have that simple ICMP check fail.

The prediciate assumption that "pinging one destination is a valid check
that my internet works' is INCORRECT. There is no magical unicorn that
could be built that could make that true, and 'they're gonna do it anyways'
is a poor excuse to even consider it.

This is a mistake many of us have made. I'll openly admit I made it 20
years ago. Like someone on the outages list I think mentioned, I had built
a couple SLA checks that triggered some routing changes to occur based on
their status, and I thought I was super hot shit. Until I had to drive an
hour through a blizzard to bring my routers back up after my incorrect
assumptions knocked my entire company (an ISP) offline. Sometimes these are
lessons people need to learn, but it's also helpful to point out to others
why what they are trying to do is a bad idea so they can( if they chose to)
learn from our prior mistakes.

On Fri, Feb 11, 2022 at 3:29 AM Mark Tinka  wrote:

>
>
> On 2/10/22 20:27, Tom Beecher wrote:
>
> >
> > I guess it depends on what the actual problem trying to be solved is.
> >
> > If I understand it correctly, the OG issue was someone (who was not
> > Google) building some monitoring around the assumption of the idea
> > that ICMP echo-request/reply to 8.8.8.8 would always be available.
> > Google decided to make a change so that assumption was now false.
> >
> > The actual problem here has nothing to do with how Google handles (or
> > doesn't handle) ICMP towards their servers. The issue is that people
> > have made poor assumptions about how they structured monitoring, and
> > learned some lessons about that. Suggesting that Party B should do
> > something because Party A made poor decisions is questionable, even if
> > it is 75% of what we do in this world.
>
> 100% - and this is the crux of the issue.
>
> As a community, it is clear that there is a need for this, and if
> 8.8.8.8 stops being an anchor for liveliness detection, users will find
> something else to replace it with. And we can bet all our Kwacha that it
> won't have been designed for that purpose, either.
>
> Mark.
>

Re: Authoritative Resources for Public DNS Pinging

[Mike Hammett]

The device that caused this whole conversation has failover functionality. Both 
interfaces ping an FQDN (that resolves to 8.8.8.8 and 1.1.1.1, with the device 
only latching on to one of those). If any of those meet the failure threshold, 
that interface is taken out of the traffic flow. 




So because someone else built a device (without a meaningful configuration to 
set otherwise), 8.8.8.8 went down for ICMP, and thus Internet ports began 
flapping, despite the Internet as a whole working just fine. 




- 
Mike Hammett 
Intelligent Computing Solutions 

Midwest Internet Exchange 

The Brothers WISP 

- Original Message -

From: "Tom Beecher"  
To: "Lady Benjamin Cannon of Glencoe"  
Cc: "NANOG Operators' Group"  
Sent: Thursday, February 10, 2022 12:27:03 PM 
Subject: Re: Authoritative Resources for Public DNS Pinging 




Seems way easier than literally everything else being proposed to me, am I 
missing something? 





I guess it depends on what the actual problem trying to be solved is. 


If I understand it correctly, the OG issue was someone (who was not Google) 
building some monitoring around the assumption of the idea that ICMP 
echo-request/reply to 8.8.8.8 would always be available. Google decided to make 
a change so that assumption was now false. 

The actual problem here has nothing to do with how Google handles (or doesn't 
handle) ICMP towards their servers. The issue is that people have made poor 
assumptions about how they structured monitoring, and learned some lessons 
about that. Suggesting that Party B should do something because Party A made 
poor decisions is questionable, even if it is 75% of what we do in this world. 







On Thu, Feb 10, 2022 at 12:52 PM Lady Benjamin Cannon of Glencoe < 
l...@6by7.net > wrote: 




Seems way easier than literally everything else being proposed to me, am I 
missing something? 


-LB 

Ms. Lady Benjamin PD Cannon of Glencoe, ASCE 
6x7 Networks & 6x7 Telecom, LLC 
CEO 
b...@6by7.net 
"The only fully end-to-end encrypted global telecommunications company in the 
world.” 
ANNOUNCING: 6x7 GLOBAL MARITIME 

FCC License KJ6FJJ 






On Feb 9, 2022, at 12:15 PM, Tom Beecher < beec...@beecher.cc > wrote: 




Side note, am I missing something obvious where I can’t just have hardware 
routers strip ICMP, pipe it separately, put 500 VMs behind 4 vLBs and let the 
world ping the brains out of it? 





Seems like a lot of overhead for zero benefit. 


On Wed, Feb 9, 2022 at 2:11 PM Lady Benjamin Cannon of Glencoe < l...@6by7.net 
> wrote: 




ok that’s amazing. 


RFC1149 amazing. 




Side note, am I missing something obvious where I can’t just have hardware 
routers strip ICMP, pipe it separately, put 500 VMs behind 4 vLBs and let the 
world ping the brains out of it? 


Who owns 69.69.69.69 - collab? 


How naff is this? 



-LB 

Ms. Lady Benjamin PD Cannon of Glencoe, ASCE 
6x7 Networks & 6x7 Telecom, LLC 
CEO 
b...@6by7.net 
"The only fully end-to-end encrypted global telecommunications company in the 
world.” 
ANNOUNCING: 6x7 GLOBAL MARITIME 

FCC License KJ6FJJ 







On Feb 9, 2022, at 9:38 AM, Jay Hennigan < j...@west.net > wrote: 


On 2/8/22 23:42, Stephane Bortzmeyer wrote: 



The only problem is the less friendly IP address (although this will 
be less and less a problem with IPv6, since 2001:4860:4860:: is 
not really friendly). 



Fun fact: Someone at Sprint had the same hobby as I did in the early 1970s. 
Their website resolves to 2600:: which I think is rather friendly. :-) 

Please don't use it for an IPv6 ping target, thanks. 

-- 
Jay Hennigan - j...@west.net 
Network Engineering - CCIE #7880 
503 897-8550 - WB6RDV 

Re: Authoritative Resources for Public DNS Pinging

[Mark Tinka]

On 2/10/22 19:42, John Todd wrote:


I think it would be fair to say that ICMP echo to easy-to-remember 
internet resources is tolerated, but not encouraged, and is probably 
not a good idea unless one knows and very well understands the 
implications of failure (or success!) modes that don’t match the 
conditions that are expected. Terrible monitoring is easy; good 
monitoring is quite difficult.


It is reasonable to expect operators of systems designed for one type 
of service to quickly rate-limit or entirely filter non-critical 
alternate capabilities in the event of resource exhaustion or other 
type of risk to the primary service - this applies to web severs, DNS 
servers, NTP servers, etc. Also, choosing as an indicator a response 
from a protocol such as ICMP echo/reply which has had a historical 
risk of flooding attacks and which may have rapid clamping of traffic 
seems to be also a large check mark in the “do not use” column. ICMP 
echo stands real risks of not providing expected results for reasons 
that are known only to the target operator, and which do not take your 
non-obvious intentions into consideration.


More central to the issue: “The Prudent Mariner never relies solely on 
any single aid to navigation” (hi, Ken!) is an applicable quote here. 
Nothing is immune from interruption of service, especially as it 
becomes more distant from your administrative control. I see all too 
often people using ICMP to a nameserver, or a query to a nameserver, 
or a socket request to port 80 of some well-known name as the only 
method utilized for determining if a larger set of systems are 
available. This is not typically a good idea. I shudder to think what 
would happen if certain well-known domains were to be unavailable due 
to one of a dozen different potential failure cases. There are far too 
many poorly-written stacks that assume some singular conditions are 
“impossible” unless as a result of local failure, and that always ends 
in sadness and late nights spent writing root-cause analysis reports.


Further adding to this complexity is the benefit or detraction of 
anycast for many of these larger public services. What is “up” and 
what is “down”? What is the signal generated or inferred by presence 
or absence of this monitoring sample? The question typically generates 
lively debate within a network or monitoring team. I am pretty sure 
that “But I could ping x.x.x.x” is not typically a statement that has 
much weight when considering overall reachability. I do admit it is a 
hint, but not the answer, for many network conditions, but probably 
not by itself should any system consider that result canonical for 
anything other than that exact result.


If one is going to use responses of exterior (not within the same 
organizational control) services as an indicator of reachability, then 
a broad spectrum of tests are probably the only way to have anything 
approaching certainty or knowledge upon which action could be based, 
and even that will always have a shadow of a doubt. In that mix, ICMP 
echo/reply to public nameservers is probably not the best indicator to 
add in a monitoring suite, though it may appear to be perfectly OK… 
until it isn’t. DNS queries to DNS servers seems to be the most 
reasonable thing to use as test material, rather than ICMP, if one 
were building a rickety monitoring house out of the resources at-hand.



Additionally: The suggestions of building some new ICMP-responding 
service may end up being counter to the goals of the people using 
external tests, so careful what is wished for. Witness everyone 
installing various “speed testing” servers in their own networks, 
which may not truly provide accurate measurements of anything other 
than local loop speeds, which now sort of defeats the purpose of the 
speed test for anything other than the most local set of results.




Well, the issue is being able to test at the lowest level possible, and 
with the lowest common denominator.


While I agree that testing an application (like DNS) makes sense, it is 
not simple, and is a lot higher up in the layers, where a lot more 
things need to work reasonably well for that test to be successful.


Ping, on the other hand, is so basic, that barring rate-limiting or 
outright blocking, is a decent indicator of liveliness between source 
and destination.


I'm all for pulling together as many tools as we can to detect 
liveliness, for I don't see Ping going anywhere, anytime soon.


Heck, e-mail has been "dying" for decades, and yet it's still here.

Mark.

Re: VPN recommendations?

[Bjørn Mork]

Sabri Berisha  writes:

> I read on some mailing list that Meraki likes to ping 8.8.8.8 every
> second... :)

That's probably to be fair with the quad-x dns providers since they
alrady were abusing 1.1.1.1.

Makes me wonder what Meraki uses 9.9.9.9 for :-)


Bjørn

Re: Authoritative Resources for Public DNS Pinging

[Mark Tinka]

On 2/10/22 22:20, Brian Knight via NANOG wrote:

On 2022-02-10 11:42, John Todd wrote:

"The Prudent Mariner never relies solely on any single aid to 
navigation"


It's best to ping multiple targets, and take action only if all 
targets do not return replies.


For the odd random ping just to see if routing works, sure.

But as a permanent monitoring technique for an IT head and their 
devices, not something we want to encourage, unless that is the business 
of the target.





For route tracking a la $VENDOR_C's IP SLA, if possible, we'll ping 
next-hop IP, one target on our network, and one off our network.  
Withdraw the default route only if all three targets fail to return 
replies.


Just put this issue into context - we have a customer who (without us 
knowing) pings our side of the p2p link. In recent days, we've had RPD 
issues (a story for another day), which has forced us to re-prioritize 
CPU resources to RIB function, and not ICMP. The customer assumed our 
network was falling over, due to the increased latency from their 
monitoring, but we cleared that up with an explanation, as revenue 
traffic is unaffected.


Mark.

Re: Authoritative Resources for Public DNS Pinging

[Mark Tinka]

On 2/10/22 20:27, Tom Beecher wrote:



I guess it depends on what the actual problem trying to be solved is.

If I understand it correctly, the OG issue was someone (who was not 
Google) building some monitoring around the assumption of the idea 
that ICMP echo-request/reply to 8.8.8.8 would always be available. 
Google decided to make a change so that assumption was now false.


The actual problem here has nothing to do with how Google handles (or 
doesn't handle) ICMP towards their servers. The issue is that people 
have made poor assumptions about how they structured monitoring, and 
learned some lessons about that. Suggesting that Party B should do 
something because Party A made poor decisions is questionable, even if 
it is 75% of what we do in this world.


100% - and this is the crux of the issue.

As a community, it is clear that there is a need for this, and if 
8.8.8.8 stops being an anchor for liveliness detection, users will find 
something else to replace it with. And we can bet all our Kwacha that it 
won't have been designed for that purpose, either.


Mark.

Re: Authoritative Resources for Public DNS Pinging

[Mark Tinka]

On 2/9/22 18:19, Joe Greco wrote:


So what people really want is to be able to "ping internet" and so far
the easiest thing people have been able to find is "ping 8.8.8.8" or
some other easily remembered thing.


Pretty much - both people and "things".



Does this mean that perhaps we should seriously consider having some
TLD being named "internet", with maybe a global DNS redirector that lets
service providers register appropriate upstream targets for their
customers, and then maybe also allow for some form of registration such
that if I wanted to provide a remote ping target for AS14536, I could
somehow register "as14536.internet" or "solnet.internet"?

Fundamentally, this is a valid issue.  As the maintainer of several BGP
networks, I can't really rely on an upstream consumer ISP to be the
connectivity helpdesk when something is awry.  It would really be nice
to have a list of officially sanctioned testing points so that one could
just do "ping google.internet" or "ping level3.internet" or "ping
comcast.internet" or "ping aws.internet" and get a response.

The problem with this is that someone will try to make what could be a
relatively simple thing complicated, and we'll end up needing a special
non-ping client and some trainwreck of names and other hard-to-grok
garbage, and then we're perilously close to coming back to the current
situation where people are using arbitrary targets out on the Internet
for connectivity testing.


Totally agree - we need to be deliberate about creating something that 
is not only simple, but memorable, in addition to being built for purpose.


But I also agree that this will likely create an opportunity to 
over-complicate what should be simple. We'll need to put in as much 
effort into resisting complexity, as we will designing a solution.


Mark.