Re: akamai yesterday - what in the world was that

[Brandon Martin]

On 2/13/20 12:39 PM, Ahmed Borno wrote:
Strictly out of interest, I wanted to ask earlier if this irresponsible 
way of causing insane, instant, bandwidth demands is breaking anything 
on the ISP/CDN side or even the console owner ?! Or is it just an 
interesting phenomenon that is handled without a sweat. Does it break 
the buck in anyway?


A good service provider will plan for things like this.  They do happen 
from time to time and for reasons other than large game updates being 
dropped in the middle of the afternoon.  However, sudden large traffic 
surges can be unexpected, causing network congestion and poor 
performance for customers, and regardless they cost money to plan for.


These game updates have become very visible recently which I suspect is 
why they're getting attention.  They've also been generating traffic 
surges during or near prime time which compounds with typical streaming 
usage and is most likely to generate customer complaints.  If they were 
hitting at 4AM local, hence the discussion about consoles and such 
downloading them automatically at night, it wouldn't be nearly as big of 
a deal since network demand is typically low during those times on 
consumer-facing networks and congestion, if it occurs, is unlikely to 
generate complaint volume.

--
Brandon Martin

Re: akamai yesterday - what in the world was that

[Ahmed Borno]

Strictly out of interest, I wanted to ask earlier if this irresponsible way
of causing insane, instant, bandwidth demands is breaking anything on the
ISP/CDN side or even the console owner ?! Or is it just an interesting
phenomenon that is handled without a sweat. Does it break the buck in
anyway?

The thread started with bandwidth surges and now power hogging is
mentioned, I wonder what else might happen as a side effect to a small
number of console/gaming companies not taking a direct responsibility in
how they release large updates in a way that is not organized or scheduled
but is rough and abrupt.

~A

On Thu, Feb 13, 2020 at 3:33 AM Tom Beecher  wrote:

> The discussion about what the consoles can or can not do is honestly not
> solving anything.
>
> Saying that the consoles should or should not be doing a thing is simply
> trying to throw the problem to someone else.
>
> On Wed, Feb 12, 2020 at 15:40 Carsten Bormann  wrote:
>
>> On 2020-02-12, at 20:45, Mike Hammett  wrote:
>> >
>> > Aren't most modern consoles on whether they're "on" or not? IE: It's
>> not a full power up from a dead stop, 0 watts power usage.
>>
>>
>> https://www.anandtech.com/show/7528/the-xbox-one-mini-review-hardware-analysis/5
>> says two-digit standby power (which they say is needed for background
>> updating).  At least in Germany, nobody sane will leave the thing in that
>> expensive mode (a watt-year is $3 here).  Switchable extension power cords
>> are being actively marketed here for these power hogs.
>>
>> Grüße, Carsten
>>
>>

RE: akamai yesterday - what in the world was that

[Aaron Gould]

I saw this ...

100 gbps inet - usually 25 gig peak - that day it was 35 gig peak

100 gbps inet - usually 25 gig peak - that day it was 35 gig peak

20 gbps (lag) inet - usually 12 gig peak - that day it was 16 gig peak

10 gig fed - aanp cluster site 1 - usually 3 gig peak - that day it sat at 10 
gig for hours (I know I was dropping packets)
 
10 gig fed - aanp cluster site 2 - usually 3 gig peak - that day it sat at 10 
gig for hours (I know I was dropping packets)

-Aaron

Re: akamai yesterday - what in the world was that

[Tom Beecher]

The discussion about what the consoles can or can not do is honestly not
solving anything.

Saying that the consoles should or should not be doing a thing is simply
trying to throw the problem to someone else.

On Wed, Feb 12, 2020 at 15:40 Carsten Bormann  wrote:

> On 2020-02-12, at 20:45, Mike Hammett  wrote:
> >
> > Aren't most modern consoles on whether they're "on" or not? IE: It's not
> a full power up from a dead stop, 0 watts power usage.
>
>
> https://www.anandtech.com/show/7528/the-xbox-one-mini-review-hardware-analysis/5
> says two-digit standby power (which they say is needed for background
> updating).  At least in Germany, nobody sane will leave the thing in that
> expensive mode (a watt-year is $3 here).  Switchable extension power cords
> are being actively marketed here for these power hogs.
>
> Grüße, Carsten
>
>

RE: Peering/Transit eBGP sessions -pet or cattle?

[adamv0025]

> Baldur Norddahl
> Sent: Wednesday, February 12, 2020 7:57 PM
> 
> On Tue, Feb 11, 2020 at 12:33 AM Lukas Tribus  wrote:
> > Therefore, if being down for several minutes is not ok, you should 
> > invest in dual links to your transits. And connect those to two 
> > different routers. If possible with a guarantee the transits use two 
> > routers at their end and that divergent fiber paths are used etc.
> 
> That is not my experience *at all*. I have always seen my prefixes 
> converge in a couple of seconds upstream (vs 2 different Tier1's).
> 
> This is a bit old but probably still thus:
> 
> https://labs.ripe.net/Members/vastur/the-shape-of-a-bgp-update
> 
> Quote: "To conclude, we observe that BGP route updates tend to 
> converge globally in just a few minutes. The propagation of newly 
> announced prefixes happens almost instantaneously, reaching 50% 
> visibility in just under 10 seconds, revealing a highly responsive 
> global system. Prefix withdrawals take longer to converge and generate 
> nearly 4 times more BGP traffic, with the visibility dropping below 10% only 
> after approximately 2 minutes".
> 
> Unfortunately they did not test the case of withdrawal from one router 
> while having the prefix still active at another.
> 
Yes that's unfortunate,
Although I'm thinking that the convergence time would be highly dependent on 
the first-hop upstream providers involved in the "local-repair" for the 
affected AS -once that is done doesn't matter that the whole world still routes 
traffic to affected AS towards the original first-hop upstream AS, as long as 
it has a valid detour route.
And I guess the topology configuration of this first-hop outskirt from the 
affected AS involved in the "local-repair" would dictate the convergence time.
E.g. if your upstream A box happens to have a direct (usable) link/session to 
upstream B box -winner, however the higher the number of boxes involved in the 
"local-repair" detour that need to be told "A no more, now B is the way to go" 
the longer the convergence time.
-but if significant portion of the Internet gets withdraw in 2 min -wondering 
how long could it be for a typical "local-repair" string of bgp speakers to all 
get the memo.
-but realistically how many bgp speakers could that be, ranging from min 2 - to 
max... say ~6? 
   


> 
> When I saw *minutes* of brownouts in connectivity it was always 
> because of ingress prefix convergence (or the lack thereof, due to 
> slow FIB programing, then temporary internal routing loops, nasty 
> things like that, but never external).
> 
> That is also a significant problem. In the case of a single transit 
> connection per router, two routers and two providers, there will be a 
> lot of internal convergence between your two routers in the case of a 
> link failure. That is also avoided by having both routers having the same 
> provider connections.
> That way a router may still have to invalidate many routes but there 
> will be no loops and the router has loop free alternatives loaded into 
> memory already (to the other provider). Plus you can use the simple 
> trick of having a default route as a fall back.
> 
This is a very good point actually, indeed since the box has two transit 
sessions in case of a failure of only one of them it will still retain all the 
prefixes in FIB -it will just need to reprogram few next-hops to point towards 
the other eBGP/iBGP speakers, whoever offers a best path. And reprograming 
next-hops is significantly faster (with hierarchical FIBs anyways).

adam

Re: Has Anyone managed to get Delegated RPKI working with ARIN

[Alex Band]

Hi there!

There is also this somewhat hacky SED command to transform the Request XML into 
the format that ARIN accepts, in case you’d like to use something other than 
the XSL:

https://sed.js.org/?gist=3f08fb293c8825855bb26f2865161575

–– Looping in John Curran

John, I appreciate ARIN has accepted RFC 8183 compatibility as an ACSP 
suggestion:

https://www.arin.net/participate/community/acsp/suggestions/2020-3/

Looking at the XML though, the changes needed to make this work are one tag, a 
URL and a version number. Could this please be tracked as a simple bug instead 
of a "feature to include in our future RPKI improvements”?

In the mean time I have added a warning to the documentation:
https://rpki.readthedocs.io/en/latest/krill/manage-cas.html#step-1-get-the-request-xml-file

Thanks!

-Alex

> On 5 Feb 2020, at 16:48, Tim Bruijnzeels  wrote:
> 
> Hi,
> 
> Everyone is welcome to read that list of course, but the TL;DR is:
> 
> ARIN currently uses a pre RFC 8183 format for the identity exchange. It would 
> be good if this were updated. New versions of rpkid as well as Krill have 
> issues with the old format.
> 
> In the meantime this XSL provided by rpki.net can be of help:
> https://raw.githubusercontent.com/dragonresearch/rpki.net/master/potpourri/oob-translate.xsl
> 
> Note: if you are planning to give Krill a try we recommend that you wait for 
> version 0.5. We expect to have this version ready in 1-2 weeks. It will 
> include usability improvements, better monitoring and a UI.
> 
> Kind regards,
> 
> Tim
> 
> 
> 
>> On 5 Feb 2020, at 16:03, Christopher Munz-Michielin  
>> wrote:
>> 
>> Brilliant! Thanks for the write up Cynthia, I'll have a read through!
>> 
>> Chris
>> 
>> On 2020-02-05 1:56 a.m., Cynthia Revström wrote:
>>> (Re-sent as I forgot to include the ML the first time, oops)
>>> Hi Chris,
>>> 
>>> I recently figured it out and posted it on the NLNetLabs RPKI mailing list. 
>>> https://lists.nlnetlabs.nl/pipermail/rpki/2020-February/000124.html 
>>> 
>>> I hope it helps :)
>>> 
>>> - Cynthia
>>> 
>>> On Wed, Jan 29, 2020 at 6:31 PM Christopher Munz-Michielin 
>>> mailto:christop...@ve7alb.ca>> wrote:
>>> 
>>>Hi Nanog,
>>> 
>>>Posting here since my Google-fu is coming up short.  I'm trying to setup 
>>> delegated RPKI in ARIN using rpki.net 's rpkid Python 
>>> daemon and am running into an issue submitting the identity file to ARIN's 
>>> control panel. The same file submitted to RIPE's  test environment at 
>>> https://localcert.ripe.net/#/rpki works without issue, while submitting to 
>>> ARIN results in "Invalid Identity.xml file."
>>> 
>>>The guide I'm following is this one: 
>>> https://github.com/dragonresearch/rpki.net/blob/master/doc/quickstart/xenial-ca.md
>>>  and I'm able to get as far as generating the identity file.
>>> 
>>>Wondering if anyone has gone down this road before and has any helpful 
>>> hints to make this work?
>>> 
>>>Cheers,
>>>Chris
>>> 
>