Re: Reminiscing our first internet connections (WAS) Re: akamai yesterday - what in the world was that

[Paul Ebersman]

kauer> When *I* were a lad we had to touch the wires with our tongues to
kauer> tell one from zero, no job for a sissy lemme tell you.

Wires? You had wires? We had to cut out our own intestines, braid them
into strands and dip them in salt water to make them conductive.

Our bosses would feed us a cup of sulphuric acid, work us in 29 hour
shifts, then kill us, and dance over our graves, singing Hallelujah.

kauer> You tell that to young folk these days and they don't believe
kauer> you...

Nope, Nope.

Damn millenials... Killing the internet...

Re: Reminiscing our first internet connections (WAS) Re: akamai yesterday - what in the world was that

[Karl Auer]

On Sat, 2020-01-25 at 22:29 -0600, Aaron Gould wrote:
> From: Ben Cannon [mailto:b...@6by7.net] 
> I started what became 6x7 with a 64k ISDN line.   And 9600 baud
> modems…   

Pah! Luxury!

When *I* were a lad we had to touch the wires with our tongues to tell
one from zero, no job for a sissy lemme tell you. And don't talk to me
about bandwidth. You could increase it easily enough by wiring up other
body parts, but it was hard to keep the staff.

My ancestors - well, up was light and down was dark, that's all *they*
knew, swimmin' around in the primordial seas. The phrase "next
generation technology" *meant* something back then.

You tell that to young folk these days and they don't believe you...

-K.

-- 
~~~
Karl Auer (ka...@biplane.com.au)
http://www.biplane.com.au/kauer
http://twitter.com/kauer389

GPG fingerprint: 8D08 9CAA 649A AFEF E862 062A 2E97 42D4 A2A0 616D
Old fingerprint: A0CD 28F0 10BE FC21 C57C 67C1 19A6 83A4 9B0B 1D75

Re: Reminiscing our first internet connections (WAS) Re: akamai yesterday - what in the world was that

[Joly MacFie]

IIRC that 64k was in fact 56k with 8k for overhead.

I had one, and it would kick in a second channel if you pushed it, for a
whopping 112k. Metered, came out to about $500/mo.

Joly

On Fri, Jan 24, 2020 at 6:26 PM Ben Cannon  wrote:

> I started what became 6x7 with a 64k ISDN line.   And 9600 baud modems…
>
> in ’93 or so.  (I was a child, in Jr High…)
>
> -Ben.
>
>
> -Ben Cannon
> CEO 6x7 Networks & 6x7 Telecom, LLC
> b...@6by7.net
>
>
>
> On Jan 24, 2020, at 3:21 PM, b...@theworld.com wrote:
>
>
> On January 24, 2020 at 08:55 aar...@gvtc.com (Aaron Gould) wrote:
>
> Thanks Jared, When I reminisce with my boss he reminds me that this
> telco/ISP here initially started with a 56kbps internet uplink , lol
>
>
> Point of History:
>
> When we, The World, first began allowing the general public onto the
> internet in October 1989 we actually had a (mildly shared*) T1
> (1.544mbps) UUNET link. So not so bad for the time. Dial-up customers
> shared a handful of 2400bps modems, we still have them.
>
> * It was also fanned out of our office to a handful of Boston-area
> customers who had 56kbps or 9600bps leased lines, not many.
>
> --
>-Barry Shein
>
> Software Tool & Die| b...@theworld.com 
> | http://www.TheWorld.com 
> Purveyors to the Trade | Voice: +1 617-STD-WRLD   | 800-THE-WRLD
> The World: Since 1989  | A Public Information Utility | *oo*
>
>
>

-- 
---
Joly MacFie  218 565 9365 Skype:punkcast
--
-

Re: akamai yesterday - what in the world was that

[Ben Cannon]

I mean I blame it on the inadequate capacity of Windstream to handle modern TCP 
traffic loads - but hey. You know. 

-Ben Cannon
CEO 6x7 Networks & 6x7 Telecom, LLC 
b...@6by7.net 




> On Jan 25, 2020, at 11:35 AM, Darin Steffl  wrote:
> 
> Shouldn't game patches like this be released overnight during off-peak hours? 
> Fortnite releases their updates around 3 or 4am when most ISP's networks are 
> at their lowest utilization. It seems somewhat reckless to release such a 
> large patch during awake hours. 
> 
> On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG  > wrote:
> "Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames outage 
> on smash-hit video game rush
> This is Windstream, going dark..."
> https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/ 
>  
> 
> Apparently not everyone came out unscathed.
> 
> --
> Brandon Jackson
> bjack...@napshome.net 
> 
> On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould  > wrote:
> My gosh, what in the word was that coming out of my local Akamai aanp servers 
> yesterday !?  starting at about 12:00 noon central time lasting several hours 
> ?
> 
>  
> 
> -Aaron
> 

RE: Dual Homed BGP

[Aaron Gould]

I’m listening to the advice of others and taking it in…. 

 

For my ISP, I’ve had 2 or 3 internet uplinks for about 12 years now for 50,000 
subs, and have only learned a default route on them.  It’s been good up to this 
point.

 

-Aaron

 

 

RE: Reminiscing our first internet connections (WAS) Re: akamai yesterday - what in the world was that

[Aaron Gould]

I love the symmetric ~10 gig speed test to put it into perspective for how far 
we’ve come….also the 3 ms ping result.  Ain’t it great

 

-Aaron

 

From: Ben Cannon [mailto:b...@6by7.net] 
Sent: Friday, January 24, 2020 5:27 PM
To: b...@theworld.com
Cc: Aaron Gould; NANOG Operators' Group
Subject: Reminiscing our first internet connections (WAS) Re: akamai yesterday 
- what in the world was that

 

I started what became 6x7 with a 64k ISDN line.   And 9600 baud modems…   

 

in ’93 or so.  (I was a child, in Jr High…)

 

-Ben.

 

 

-Ben Cannon

CEO 6x7 Networks & 6x7 Telecom, LLC 

b...@6by7.net

 




 

On Jan 24, 2020, at 3:21 PM, b...@theworld.com wrote:

 


On January 24, 2020 at 08:55 aar...@gvtc.com (Aaron Gould) wrote:



Thanks Jared, When I reminisce with my boss he reminds me that this telco/ISP 
here initially started with a 56kbps internet uplink , lol


Point of History:

When we, The World, first began allowing the general public onto the
internet in October 1989 we actually had a (mildly shared*) T1
(1.544mbps) UUNET link. So not so bad for the time. Dial-up customers
shared a handful of 2400bps modems, we still have them.

* It was also fanned out of our office to a handful of Boston-area
customers who had 56kbps or 9600bps leased lines, not many.

-- 
   -Barry Shein

Software Tool & Die| b...@theworld.com | http://www.TheWorld.com
Purveyors to the Trade | Voice: +1 617-STD-WRLD   | 800-THE-WRLD
The World: Since 1989  | A Public Information Utility | *oo*

 

Re: akamai yesterday - what in the world was that

[Brandon Jackson via NANOG]

Xbox One has 2 options, always one (equivalent to windows sleep) and will
wake up occasionally for updates, and power save (equivalent to hibernate
ish) it will not wake up for updates.


Brandon Jackson
bojack1...@gmail.com
478-387-8687

On Sat, Jan 25, 2020, 21:51 Mike Hammett  wrote:

> IIRC, game consoles are always on, whether they're "on" or not.
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> --
> *From: *"Tom Beecher" 
> *To: *"Darin Steffl" 
> *Cc: *Nanog@nanog.org
> *Sent: *Saturday, January 25, 2020 5:13:19 PM
> *Subject: *Re: akamai yesterday - what in the world was that
>
> Not everybody leaves their console/PC on 24/7 so that they would pull the
> patch at 3am local even if that’s when it was released.
>
> It’s far from reckless. It’s not the game companies job to make sure the
> network works. That’s our job.
>
> On Sat, Jan 25, 2020 at 14:37 Darin Steffl 
> wrote:
>
>> Shouldn't game patches like this be released overnight during off-peak
>> hours? Fortnite releases their updates around 3 or 4am when most ISP's
>> networks are at their lowest utilization. It seems somewhat reckless to
>> release such a large patch during awake hours.
>>
>> On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG 
>> wrote:
>>
>>> "Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames
>>> outage on smash-hit video game rush
>>> This is Windstream, going dark..."
>>> https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/
>>>
>>> Apparently not everyone came out unscathed.
>>>
>>> --
>>> Brandon Jackson
>>> bjack...@napshome.net
>>>
>>>
>>> On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould  wrote:
>>>
 My gosh, what in the word was that coming out of my local Akamai aanp
 servers yesterday !?  starting at about 12:00 noon central time lasting
 several hours ?



 -Aaron

>>>
>

Re: akamai yesterday - what in the world was that

[Mike Hammett]

IIRC, game consoles are always on, whether they're "on" or not. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 

Midwest-IX 
http://www.midwest-ix.com 

- Original Message -

From: "Tom Beecher"  
To: "Darin Steffl"  
Cc: Nanog@nanog.org 
Sent: Saturday, January 25, 2020 5:13:19 PM 
Subject: Re: akamai yesterday - what in the world was that 



Not everybody leaves their console/PC on 24/7 so that they would pull the patch 
at 3am local even if that’s when it was released. 


It’s far from reckless. It’s not the game companies job to make sure the 
network works. That’s our job. 



On Sat, Jan 25, 2020 at 14:37 Darin Steffl < darin.ste...@mnwifi.com > wrote: 



Shouldn't game patches like this be released overnight during off-peak hours? 
Fortnite releases their updates around 3 or 4am when most ISP's networks are at 
their lowest utilization. It seems somewhat reckless to release such a large 
patch during awake hours. 


On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG < nanog@nanog.org > 
wrote: 



"Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames outage 
on smash-hit video game rush 
This is Windstream, going dark..." 
https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/ 


Apparently not everyone came out unscathed. 






-- 
Brandon Jackson 
bjack...@napshome.net 



On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould < aar...@gvtc.com > wrote: 





My gosh, what in the word was that coming out of my local Akamai aanp servers 
yesterday !? starting at about 12:00 noon central time lasting several hours ? 

-Aaron 

Re: Data on latency and loss-rates during congestion DDoS attacks

[Amir Herzberg]

Hi Damian, thanks, that's right; actually in high-latency and 10% loss, you
get _much_ better performance than either TCP or Quic. However, these are
not as common scenarios as clogging due to DDoS... So we still want to find
relevant data, to know which ranges of latency and loss make sense.

Guys: if you can share data but only privately, please do :) thanks!

Amir

-- 
Amir



On Sat, Jan 25, 2020 at 12:38 PM Damian Menscher  wrote:

> Getting (and releasing) numbers from DDoS attacks will be challenging for
> most, but I think your research could apply to more than just DDoS.  There
> are often cases where one might want to work from an environment which has
> very poor networking.  As an extreme example, in 2007 I got online from an
> internet cafe in Paramaribo.  But, as I told a friend at the time, "latency
> is about 1s and packet loss around 10%".  It would be great if forward
> error correction could have improved that experience.
>
> Damian
>
> On Fri, Jan 24, 2020 at 7:27 PM Amir Herzberg 
> wrote:
>
>> Damian, thanks!
>>
>> That's actually roughly the range of losses we focused on; but it was
>> based on my rough feeling for reasonable loss rates (as well as on
>> experiments where we caused losses in emulated environments), and a
>> reviewer - justifiably - asked if we can base our values on realistic
>> values. So I would love to have real value, I'm sure some people have these
>> measured (I'm actually quite sure I've seed such values, but the challenge
>> is recalling where and finding it...).
>>
>> Also, latency values (under congestion) would be appreciated. Also here,
>> we used a range of values, I think the highest was 1sec, since we believe
>> that under congestion delays goes up considerably since many queues fill up
>> [and again I seem to recall values around this range]. But here the
>> reviewer even challenged us and said he/she doubts that delays increase
>> significantly under network congestion since he/she thinks that the
>> additional queuing is something mostly in small routers such as home
>> routers (and maybe like the routers used in our emulation testbed). So I'll
>> love to have some real data to know for sure.
>>
>> Apart from knowing these things for this specific paper, I should know
>> them in a well-founded way anyway, as I'm doing rearch on and teaching
>> net-sec (incl. quite a lot on DoS) :)
>>
>> --
>> Amir
>>
>>
>>
>> On Fri, Jan 24, 2020 at 5:31 PM Damian Menscher 
>> wrote:
>>
>>> I suggest testing with a broad variety of values, as losses as low as 5%
>>> can be annoying, but losses at 50% or more are not uncommon.
>>>
>>> Damian
>>>
>>> On Fri, Jan 24, 2020 at 4:41 AM Amir Herzberg 
>>> wrote:
>>>
 Dear NANOG,

 One of my ongoing research works is about a transport protocol that
 ensures (critical) communication in spite of DDoS congestion attack (which
 cannot be circumvented), by (careful) use of Forward Error Correction. Yes,
 obviously, this has to be done and used carefully since the FEC clearly
 increases traffic rather than the typical congestion-control approach of
 reducing it, I'm well aware of it; but some applications are critical (and
 often low-bandwidth) so such tool is important.

 I am looking for data on loss rate and congestion of DDoS attacks to
 make sure we use right parameters. Any chance you have such data and can
 share?

 Many thanks!
 --
 Amir Herzberg
 Comcast chair of security innovation, University of Connecticut
 Foundations of cybersecurity
 ,
  part
 I (see also part II and presentations):
 https://www.researchgate.net/publication/323243320_Introduction_to_Cyber-Security_Part_I_Applied_Cryptography_Lecture_notes_and_exercises
 

Re: akamai yesterday - what in the world was that

[Tom Beecher]

Not everybody leaves their console/PC on 24/7 so that they would pull the
patch at 3am local even if that’s when it was released.

It’s far from reckless. It’s not the game companies job to make sure the
network works. That’s our job.

On Sat, Jan 25, 2020 at 14:37 Darin Steffl  wrote:

> Shouldn't game patches like this be released overnight during off-peak
> hours? Fortnite releases their updates around 3 or 4am when most ISP's
> networks are at their lowest utilization. It seems somewhat reckless to
> release such a large patch during awake hours.
>
> On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG 
> wrote:
>
>> "Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames
>> outage on smash-hit video game rush
>> This is Windstream, going dark..."
>> https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/
>>
>> Apparently not everyone came out unscathed.
>>
>> --
>> Brandon Jackson
>> bjack...@napshome.net
>>
>>
>> On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould  wrote:
>>
>>> My gosh, what in the word was that coming out of my local Akamai aanp
>>> servers yesterday !?  starting at about 12:00 noon central time lasting
>>> several hours ?
>>>
>>>
>>>
>>> -Aaron
>>>
>>

Re: akamai yesterday - what in the world was that

[J. Hellenthal via NANOG]

That’s what she said

-- 
 J. Hellenthal

The fact that there's a highway to Hell but only a stairway to Heaven says a 
lot about anticipated traffic volume.

> On Jan 25, 2020, at 13:42, Alistair Mackenzie  wrote:
> 
> 
> Off-peak hours are on-peak somewhere else in the world. 
> 
>> On Sat, Jan 25, 2020 at 7:37 PM Darin Steffl  wrote:
>> Shouldn't game patches like this be released overnight during off-peak 
>> hours? Fortnite releases their updates around 3 or 4am when most ISP's 
>> networks are at their lowest utilization. It seems somewhat reckless to 
>> release such a large patch during awake hours. 
>> 
>>> On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG  
>>> wrote:
>>> "Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames 
>>> outage on smash-hit video game rush
>>> This is Windstream, going dark..."
>>> https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/ 
>>> 
>>> Apparently not everyone came out unscathed.
>>> 
>>> --
>>> Brandon Jackson
>>> bjack...@napshome.net
>>> 
>>> 
 On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould  wrote:
 My gosh, what in the word was that coming out of my local Akamai aanp 
 servers yesterday !?  starting at about 12:00 noon central time lasting 
 several hours ?
 
  
 
 -Aaron


smime.p7s
Description: S/MIME cryptographic signature

Re: akamai yesterday - what in the world was that

[Alistair Mackenzie]

Off-peak hours are on-peak somewhere else in the world.

On Sat, Jan 25, 2020 at 7:37 PM Darin Steffl 
wrote:

> Shouldn't game patches like this be released overnight during off-peak
> hours? Fortnite releases their updates around 3 or 4am when most ISP's
> networks are at their lowest utilization. It seems somewhat reckless to
> release such a large patch during awake hours.
>
> On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG 
> wrote:
>
>> "Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames
>> outage on smash-hit video game rush
>> This is Windstream, going dark..."
>> https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/
>>
>> Apparently not everyone came out unscathed.
>>
>> --
>> Brandon Jackson
>> bjack...@napshome.net
>>
>>
>> On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould  wrote:
>>
>>> My gosh, what in the word was that coming out of my local Akamai aanp
>>> servers yesterday !?  starting at about 12:00 noon central time lasting
>>> several hours ?
>>>
>>>
>>>
>>> -Aaron
>>>
>>

Re: akamai yesterday - what in the world was that

[Darin Steffl]

Shouldn't game patches like this be released overnight during off-peak
hours? Fortnite releases their updates around 3 or 4am when most ISP's
networks are at their lowest utilization. It seems somewhat reckless to
release such a large patch during awake hours.

On Sat, Jan 25, 2020, 12:08 PM Brandon Jackson via NANOG 
wrote:

> "Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames
> outage on smash-hit video game rush
> This is Windstream, going dark..."
> https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/
>
> Apparently not everyone came out unscathed.
>
> --
> Brandon Jackson
> bjack...@napshome.net
>
>
> On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould  wrote:
>
>> My gosh, what in the word was that coming out of my local Akamai aanp
>> servers yesterday !?  starting at about 12:00 noon central time lasting
>> several hours ?
>>
>>
>>
>> -Aaron
>>
>

Re: akamai yesterday - what in the world was that

[Brandon Jackson via NANOG]

"Call of Duty: Modern Warfare fragged our business VOIP: US ISP blames
outage on smash-hit video game rush
This is Windstream, going dark..."
https://www.theregister.co.uk/2020/01/23/windstream_fvoip_outage/

Apparently not everyone came out unscathed.

--
Brandon Jackson
bjack...@napshome.net


On Thu, Jan 23, 2020 at 10:14 AM Aaron Gould  wrote:

> My gosh, what in the word was that coming out of my local Akamai aanp
> servers yesterday !?  starting at about 12:00 noon central time lasting
> several hours ?
>
>
>
> -Aaron
>

Re: Data on latency and loss-rates during congestion DDoS attacks

[Damian Menscher via NANOG]

Getting (and releasing) numbers from DDoS attacks will be challenging for
most, but I think your research could apply to more than just DDoS.  There
are often cases where one might want to work from an environment which has
very poor networking.  As an extreme example, in 2007 I got online from an
internet cafe in Paramaribo.  But, as I told a friend at the time, "latency
is about 1s and packet loss around 10%".  It would be great if forward
error correction could have improved that experience.

Damian

On Fri, Jan 24, 2020 at 7:27 PM Amir Herzberg  wrote:

> Damian, thanks!
>
> That's actually roughly the range of losses we focused on; but it was
> based on my rough feeling for reasonable loss rates (as well as on
> experiments where we caused losses in emulated environments), and a
> reviewer - justifiably - asked if we can base our values on realistic
> values. So I would love to have real value, I'm sure some people have these
> measured (I'm actually quite sure I've seed such values, but the challenge
> is recalling where and finding it...).
>
> Also, latency values (under congestion) would be appreciated. Also here,
> we used a range of values, I think the highest was 1sec, since we believe
> that under congestion delays goes up considerably since many queues fill up
> [and again I seem to recall values around this range]. But here the
> reviewer even challenged us and said he/she doubts that delays increase
> significantly under network congestion since he/she thinks that the
> additional queuing is something mostly in small routers such as home
> routers (and maybe like the routers used in our emulation testbed). So I'll
> love to have some real data to know for sure.
>
> Apart from knowing these things for this specific paper, I should know
> them in a well-founded way anyway, as I'm doing rearch on and teaching
> net-sec (incl. quite a lot on DoS) :)
>
> --
> Amir
>
>
>
> On Fri, Jan 24, 2020 at 5:31 PM Damian Menscher  wrote:
>
>> I suggest testing with a broad variety of values, as losses as low as 5%
>> can be annoying, but losses at 50% or more are not uncommon.
>>
>> Damian
>>
>> On Fri, Jan 24, 2020 at 4:41 AM Amir Herzberg 
>> wrote:
>>
>>> Dear NANOG,
>>>
>>> One of my ongoing research works is about a transport protocol that
>>> ensures (critical) communication in spite of DDoS congestion attack (which
>>> cannot be circumvented), by (careful) use of Forward Error Correction. Yes,
>>> obviously, this has to be done and used carefully since the FEC clearly
>>> increases traffic rather than the typical congestion-control approach of
>>> reducing it, I'm well aware of it; but some applications are critical (and
>>> often low-bandwidth) so such tool is important.
>>>
>>> I am looking for data on loss rate and congestion of DDoS attacks to
>>> make sure we use right parameters. Any chance you have such data and can
>>> share?
>>>
>>> Many thanks!
>>> --
>>> Amir Herzberg
>>> Comcast chair of security innovation, University of Connecticut
>>> Foundations of cybersecurity
>>> ,
>>>  part
>>> I (see also part II and presentations):
>>> https://www.researchgate.net/publication/323243320_Introduction_to_Cyber-Security_Part_I_Applied_Cryptography_Lecture_notes_and_exercises
>>> 
>>>
>>>
>>>

Re: akamai yesterday - what in the world was that (now old guy stuff)

[Allen McKinley Kitchen (gmail)]

On Jan 25, 2020, at 08:52, Paul Nash  wrote:
> 
> 
>> 
>> So, I grew up in South Africa, and one of the more fascinating /
>> cooler things I saw was a modem which would get you ~50bps (bps, not
>> Kbps) over a single strand of barbed wire -- you'd hammer a largish
>> nail into the ground, and clip one alligator[0] clip onto that, and
>> another alligator clip onto the barbed wire. Repeat the process on the
>> other side (up to ~5km away), plug the modems in, and bits would
>> flow... I only saw these used a few times, but always thought they
>> were cool….
> 
> Do you remember anything about the actual type of modem?  Or where you 
> deployed them?
> 
Decades ago, I cobbled together a 20mA current loop interface that may have 
been an early version of this .. ran a set of Baudot machines (pre-ASCII, upper 
case & figs only) mostly just to have fun with a set of old ASR 32 teletypes.  
I used a couple of 500’ spools of zip cord lying on the ground from end to end. 
Never mind backhoes - it was lawn-mower vulnerable. (However, being flat on the 
ground seemingly made it less vulnerable to lightning strikes.)

Of course, this was hardly critical infrastructure!

Blessings..

Allen

Re: Dual Homed BGP

[Baldur Norddahl]

lør. 25. jan. 2020 13.42 skrev Tore Anderson :

> * Baldur Norddahl
>
> > If you join any peering exchanges, full tables will be mandatory. Some
> parties will export prefixes and then expect a more specific prefix
> received from your transit to override a part of the space received via the
> peering.
>
> That would be a fundamentally flawed expectation, in my opinion.
>

I do not disagree, however the real world sometimes works differently. Like
anycast, people break the rules and gets away with it.

In any case, this is from a recent personal experience. We had a problem
that led us to drop full tables and run with a default for a while. Nobody
noticed the difference, which is why I can confidently say that unless you
need the full tables for something, the advantages are somewhat overstated.

However one customer found a reachability problem. Turns out that a peer
was exporting a /19 prefix through a peering session with us and at another
site they exported a /24 from the same space with no routing between sites.

Regards

Baldur

>

Re: akamai yesterday - what in the world was that

[Paul Nash]

> So, I grew up in South Africa, and one of the more fascinating /
> cooler things I saw was a modem which would get you ~50bps (bps, not
> Kbps) over a single strand of barbed wire -- you'd hammer a largish
> nail into the ground, and clip one alligator[0] clip onto that, and
> another alligator clip onto the barbed wire. Repeat the process on the
> other side (up to ~5km away), plug the modems in, and bits would
> flow... I only saw these used a few times, but always thought they
> were cool….

Do you remember anything about the actual type of modem?  Or where you deployed 
them?

In the days before the Internet came to SA, I ran a dial-up email link between 
the US and Pretoria, polled by various people locally (including CSIR, SAIMR).  
I also carried mail for the UNHCR in Northern Mozambique.  Mail came via Karl 
Deninger (DDSW1) in Chicago, IIRC.

They were missing several kilometres of phone wire, so connected the link to 
the fence on each side of the road.  We get about 1200bps on a good day IIRC, 
and would loose carrier whenever someone moved cattle from one field to another 
and opened a gate in the fence.

paul

Re: Dual Homed BGP

[Tore Anderson]

* Baldur Norddahl

> If you join any peering exchanges, full tables will be mandatory. Some 
> parties will export prefixes and then expect a more specific prefix received 
> from your transit to override a part of the space received via the peering. 

That would be a fundamentally flawed expectation, in my opinion.

An AS that that advertises a prefix to its peers must be prepared to carry 
traffic to that entire prefix via that peering circuit.

There is simply no guarantee that a more-specific prefix advertised somewhere 
else will make it into the RIBs and FIBs of all the peers of that AS.

The AS might of course opt to do so anyway for traffic engineering purposes, 
but there is no assurance that it will actually work 100% of the time. When it 
doesn't, the AS in question would need to carry the traffic from the peering 
circuit across their own backbone.

If the AS in question for some reason cannot do so, it would need to adjust its 
advertisements across the peering circuit so as to avoid falsely advertising 
reachability to unreachable destinations.

Tore

Re: Rogue objects in routing databases

[Florian Brandstetter]

Hi Martijn,

albeit a negligible amount of edge cases it can
indeed be stated that there is too much trust put
into alternative IRR sources operated by third
partys not affiliated to RIRs. Generally, usage of
such databases however is not mis-used in a larger
scope, and the complexity involved with creating
route objects (AltDB for example validates new
MAINTAINERS, RaDB charges) diminishes the vector
in a (barely influental) manner. An option to
combat this would perhaps be to run validation at
regular intervals, and brings invalid objects to
the attention of operators. I did similar for this
incident just a moment ago with a batch of self-
written scripts.

After further analysis, there are over 5390 IPv4
prefixes pointing with their origin towards
AS8100: https://pastebin.com/Zh1YZfEq

Out of 5390 prefixes, 2287 are currently not even
visible within the global routing table: 
https://pastebin.com/cSepb7yS

Another 2714 prefixes are INVALID, that in
particular means that AS8100 is neither *within*
the announcing AS-PATH, nor originating the
prefix: https://pastebin.com/JhaxVeN0

Last but not least, there is 389 VALID prefixes
(in this case, perhaps only technically valid, as
I did probe for AS8100 within the AS-PATH
sequence, and not if AS8100 actually originates
the prefix): https://pastebin.com/UVt6nwGz

That's a conceivable 5001 IPv4 prefixes for a
potential bad actor right there. It can also
clearly be stated that, while initially mentioned,
the significance of ascendence caused by AS-SBAG
is negligible, as it appears, the entirity of
Quadranet and affiliates is affected.

Regards,
Florian Brandstetter

On Sat, 2020-01-25 at 01:02 +, Martijn Schmidt
wrote:
> Hi Florian, NANOG, 
> 
> While the symptom of (automatically) proxy
> registered route objects is problematic, perhaps
> we could also take this opportunity to discuss
> the underlying issue: we as an industry appear
> to place our trust in various IRR sources
> operated by entities that either can't or don't
> validate whether the actual owner of the
> involved resource approves the creation of the
> IRR database object.
> 
> We should start to push our customers to
> maintain their route origin information in
> databases operated by the RIR or NIR which
> assigned the resource, or even through RPKI ROAs
> that were optionally converted into IRR route
> objects for the ease of consumption. It's also
> time for the RIRs to take their responsibility
> in this matter by facilitating services like
> IRR, RPKI, PTR, etc for legacy IP space under
> conditions which are palatable to corporate
> lawyers, if they haven't already done so. 
> 
> Finally, there doesn't have to be a global "flip
> the switch" day where we decide to stop trusting
> 3rd party databases, but even if we start
> holding ourselves to a higher standard one
> customer at a time that's still going to have
> the potential to make a big difference a couple
> of years down the road. 
> 
> Best regards, 
> Martijn Schmidt 
> 
> PS, a small disclaimer: none of the above are
> new ideas, nor did I come up with them myself -
> but it still makes sense to work towards
> implementing them.. 
> From: NANOG  on behalf
> of Florian Brandstetter 
> Sent: 25 January 2020 00:06
> To: nanog@nanog.org 
> Subject: Rogue objects in routing databases
>  
> It appears that there is currently an influx of
> rogue route
> objects created within the NTTCOM and RaDB IRR
> databases, in
> connection to Quadranet (AS8100) and China
> Mobile
> International (CMI).
> 
> Examples of affected networks are:
> 
> 193.30.32.0/23
> 45.129.92.0/23
> 45.129.94.0/24
> 
> Networks, which have seemingly no affiliation
> with
> Quadranet, nor China Mobile International (CMI),
> which
> merely appears to be an upstream of Quadranet
> and hence
> creates the route objects in an automated
> manner.
> 
> Another person has already reached out to
> Quadranet to find
> out the root cause of the creation of these
> objects. Their
> support gave an ETA of 24-72 hours.
> 
> The route objects are all identical:
> 
> route:  193.30.32.0/23
> descr:  CMI  (Customer Route)
> origin: AS8100
> mnt-by: MAINT-AS58453
> changed:qas_supp...@cmi.chinamobile.com
> 20200117
> source: RADB
> 
> There appears to be a correlation with the
> affected
> networks, a fair share of them is part of AS-
> SBAG, which in
> turn is part of AS-VMHAUS, which in turn is part
> of AS-
> QUADRANET and could yield the importing of these
> prefixes.
> AS-VMHAUS appears to be a customer of Quadranet,
> listed
> within AS-QUADRANET-CUSTOMER-ASSET.
> 
> These networks do however have no direct
> connection to
> Quadranet, and are not affiliated with
> Quadranet, nor are
> currently connected to Quadranet, which,
> entirely ignoring
> that the `origin` points to Quadranet, makes the
> route
> object illicit.
> 
> Basically this has given AS8100, whether that be
> legitimately Quadranet, or somebody
> impersonating/spinning
> up a rogue 

Re: Data on latency and loss-rates during congestion DDoS attacks

[Amir Herzberg]

On Sat, Jan 25, 2020 at 2:12 AM Saku Ytti  wrote:

> On Sat, 25 Jan 2020 at 05:30, Amir Herzberg  wrote:
>
> DDoS is very very cheap, if there is a single global egress for given
> interface then the DDoS traffic can easily be 100 times the egress
> capacity (1GE egress, 100GE DDoS).


Thanks. However, my question is about statistics of attacks actually seen
`in the wild' - and not just the `worst' but also more common attacks.
Furthermore, I'm asking about the _outcome_ of the congestion - mainly,
loss-rates and latency - and not about the _amount_ of DDoS traffic. DDoS
traffic often gets lost itself in different intermediate routers, so its
ultimate impact is not trivial to estimate.


> I'm very skeptical if FEC will
> help, I think this is case of cat and mouse


hmm, I don't think so; it is more a matter of justification, and also,
obviously, amount of over-capacity - which is still, obviously, a basic
thing anybody concerned about congestion would worry about. Let me be
extreme and simplify... Suppose idd attacker can send 100 times the
capacity of a (say, single) router, resulting in 99% loss rate. Then FEC
should work - but, of course, with high overhead, let's even simplify and
say it requires 100 times redundancy (although it's actually not as bad as
that). Still, this can be Ok if I have 100 times overcapacity - which for
many critical applications, is not even a big deal, as crazy as it sounds
(and is) for general applications.


> , based on data you see now
> it may seem reasonable, but now is only result of minimum viable ddos,
> which is trivial to increase should need occur.


I still think evaluation should preferably compare to attacks reported in
reality, with potential additional analysis of projections of potential
attacks.


> Similarly DDoS attacks
> are excessive dumb often, like dumb UDP ports which are easy drop, but
> should we solve protection well for these, it's trivial to make it
> proper HTTPS TCP SYN.
>

hmm, tcp-syn is already a different story (and we have pretty good defenses
against it and many other attacks on the end host). I do work on some of
these attacks (and defenses) too but in this specific case I'm focusing on
bandwidth-DoS attacks (network congestion).  I'm further focusing in this
work on a defense which may involves a transport (end to end) protocol, of
course I'm aware of network-based defenses, it's just not focus of this
work (think of customer with no ability to `fix' the network service).

>
> Backbone device interface can add hundreds of milliseconds during
> congestion, but more commonly we're talking about tens of milliseconds
> during congestion and low microseconds to high nanoseconds outside
> congestion.
> Backbone device buffer space is highly googlable, BRCM
> trident/tomahawk styte boxes have very little, but they are more
> intended for DC/LAN scenarios, than WAN. Nokia FP, Huawei Solar,
> EZchip, Cisco nPower, Cisco Silicon One, Juniper Trio, Juniper
> Paradise, Broadcom Jericho all will buffer high tens of milliseconds
> to low hundreds.
>

Thanks again, but I'm not looking for data on particular devices; the
latency during congestion attacks may be impacted by multiple devices along
the path. So again my interest is mainly in measured values under real
attacks.

tks! Amir

>
> --
>   ++ytti
>

Re: akamai yesterday - what in the world was that

[Nick Hilliard]

Valdis Klētnieks wrote on 24/01/2020 21:20:

I remember when a "gateway" was a Microvax II with an ethernet card and a
bisync card


I remember the day when the microvax II and all the other vaxes on 
campus were upgraded from CMU-TEK to the Multinet TCP/IP stack.  Gone 
were the days of maxing out at 14kbit/s on the high speed 10 megabit 
co-ax campus backbone - we could now get hundreds of kilobits per 
second.  The joy of it.


That decision to upgrade happened the day that all CMU-TEK installations 
in the world went offline due to some bug or other.


Nick