Re: Famous operational issues

[Mark Tinka]

Hardly famous and not service-affecting in the end, but figured I'd 
share an incident from our side that occurred back in 2018.


While commissioning a new node in our Metro-E network, an IPv6 
point-to-point address was mis-typed. Instead of ending in /126, it 
ended in /12. This happened in Johannesburg.


We actually came across this by chance while examining the IGP table of 
another router located in Slough, and found an entry for 2c00::/12 
floating around. That definitely looked out of place, as we never carry 
parent blocks in our IGP.


Running the trace from Slough led us back to this one Metro-E device in 
Jo'burg.


It took everyone nearly an hour to figure out the typo, because for all 
the laser focus we had on the supposed link of the supposed box that was 
creating this problem, we all overlooked the fact that the /12 
configured on the point-to-point link was actually supposed to have been 
a /126.


The reason this never caused a service problem was because we do not 
redistribute our IGP into BGP (not that anyone should). And even if we 
did, there are a ton of filters and BGP communities on all devices to 
ensure a route such as that would have never made it out of our AS.


Also, the IGP contains the most specific paths to every node in our 
network, so the presence of the 2c00::/12 was mostly cosmetic. It would 
have never been used for routing decisions.


Mark.

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Rich Kulawiec]

On Fri, Mar 12, 2021 at 02:46:51PM +, David Hubbard wrote:
> After sending them abuse reports for years with only an increase in
> malicious traffic, I have no expectation of anything they do getting
> better or being for the benefit of the internet as a whole.

This is a shared experience. It's abundantly clear that OVH is either
acting in concert with the abusers, or they *are* the abusers.  It doesn't
really matter which, the operational outcome is the same in either case.

The best course of action is to remove them from your (generic you)
view of the Internet via whatever means are most expedient.

---rsk

Seattle/LA 100 Gig Wave

[Rod Beck]

A point: Westin House, Seattle.
Z point: 1 Wilshire, LA.
Routing: Seattle/Boise/Ogden/Salt Lake City/LA.
Term: 3 years.

Need help on this circuit. Contact me off-list.

Regards,

Roderick.



Roderick Beck

Global Network Capacity Procurement

United Cable Company

www.unitedcablecompany.com
https://unitedcablecompany.com/video/
New York City & Budapest

rod.b...@unitedcablecompany.com

Budapest: 36-70-605-5144

NJ: 908-452-8183



[1467221477350_image005.png]

Weekly Routing Table Report

[Routing Analysis Role Account]

This is an automated weekly mailing describing the state of the Internet
Routing Table as seen from APNIC's router in Japan.

The posting is sent to APOPS, NANOG, AfNOG, SANOG, PacNOG, SAFNOG
TZNOG, MENOG, BJNOG, SDNOG, CMNOG, LACNOG and the RIPE Routing WG.

Daily listings are sent to bgp-st...@lists.apnic.net

For historical data, please see http://thyme.rand.apnic.net.

If you have any comments please contact Philip Smith .

Routing Table Report   04:00 +10GMT Sat 13 Mar, 2021

Report Website: http://thyme.rand.apnic.net
Detailed Analysis:  http://thyme.rand.apnic.net/current/

Analysis Summary


BGP routing table entries examined:  849542
Prefixes after maximum aggregation (per Origin AS):  322940
Deaggregation factor:  2.63
Unique aggregates announced (without unneeded subnets):  405118
Total ASes present in the Internet Routing Table: 70782
Prefixes per ASN: 12.00
Origin-only ASes present in the Internet Routing Table:   60922
Origin ASes announcing only one prefix:   25123
Transit ASes present in the Internet Routing Table:9860
Transit-only ASes present in the Internet Routing Table:296
Average AS path length visible in the Internet Routing Table:   4.3
Max AS path length visible:  47
Max AS path prepend of ASN ( 17747)  41
Prefixes from unregistered ASNs in the Routing Table:   996
Number of instances of unregistered ASNs:  1000
Number of 32-bit ASNs allocated by the RIRs:  35352
Number of 32-bit ASNs visible in the Routing Table:   29397
Prefixes from 32-bit ASNs in the Routing Table:  136778
Number of bogon 32-bit ASNs visible in the Routing Table:27
Special use prefixes present in the Routing Table:1
Prefixes being announced from unallocated address space:538
Number of addresses announced to Internet:   2916667008
Equivalent to 173 /8s, 216 /16s and 206 /24s
Percentage of available address space announced:   78.8
Percentage of allocated address space announced:   78.8
Percentage of available address space allocated:  100.0
Percentage of address space in use by end-sites:   99.5
Total number of prefixes smaller than registry allocations:  289248

APNIC Region Analysis Summary
-

Prefixes being announced by APNIC Region ASes:   223226
Total APNIC prefixes after maximum aggregation:   65172
APNIC Deaggregation factor:3.43
Prefixes being announced from the APNIC address blocks:  219346
Unique aggregates announced from the APNIC address blocks:89061
APNIC Region origin ASes present in the Internet Routing Table:   11360
APNIC Prefixes per ASN:   19.31
APNIC Region origin ASes announcing only one prefix:   3234
APNIC Region transit ASes present in the Internet Routing Table:   1602
Average APNIC Region AS path length visible:4.5
Max APNIC Region AS path length visible: 47
Number of APNIC region 32-bit ASNs visible in the Routing Table:   6510
Number of APNIC addresses announced to Internet:  770758400
Equivalent to 45 /8s, 240 /16s and 215 /24s
APNIC AS Blocks4608-4864, 7467-7722, 9216-10239, 17408-18431
(pre-ERX allocations)  23552-24575, 37888-38911, 45056-46079, 55296-56319,
   58368-59391, 63488-64098, 64297-64395, 131072-143673
APNIC Address Blocks 1/8,  14/8,  27/8,  36/8,  39/8,  42/8,  43/8,
49/8,  58/8,  59/8,  60/8,  61/8, 101/8, 103/8,
   106/8, 110/8, 111/8, 112/8, 113/8, 114/8, 115/8,
   116/8, 117/8, 118/8, 119/8, 120/8, 121/8, 122/8,
   123/8, 124/8, 125/8, 126/8, 133/8, 150/8, 153/8,
   163/8, 171/8, 175/8, 180/8, 182/8, 183/8, 202/8,
   203/8, 210/8, 211/8, 218/8, 219/8, 220/8, 221/8,
   222/8, 223/8,

ARIN Region Analysis Summary


Prefixes being announced by ARIN Region ASes:244378
Total ARIN prefixes after maximum aggregation:   112352
ARIN Deaggregation factor: 2.18
Prefixes being announced from the ARIN address blocks:   245021
Unique aggregates announced from the ARIN address blocks:116945
ARIN Region origin ASes present in the Internet Routing Table:18742
ARIN Prefixes per ASN:13.07
ARIN 

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Alain Hebert]

    One hope that the IPO will bring more pressure to evaluate the 
gain/loss ratio of handling the abuses to the community satisfaction.


    ( As for the the fire, I have to feel bad for all the staff working 
endless hours to get their customers back online ).


-
Alain Hebertaheb...@pubnix.net
PubNIX Inc.
50 boul. St-Charles
P.O. Box 26770 Beaconsfield, Quebec H9W 6G7
Tel: 514-990-5911  http://www.pubnix.netFax: 514-990-9443

On 3/12/21 10:19 AM, ke...@contoocook.net wrote:
That's the first mention of abuse I've seen. I've got huge blocks of 
OVH space blocked.. Lots of bad things coming from there. Not sure the 
fire will change things down the road as the sources are from all over 
the roadmap.


Sent from MailDroid 

-Original Message-
From: eric-l...@truenet.com
To: nanog@nanog.org
Sent: Fri, 12 Mar 2021 9:54
Subject: RE: OVH datacenter SBG2 in Strasbourg on fire 

> -Original Message-
> From: NANOG > On Behalf Of David Hubbard

> Sent: Friday, March 12, 2021 9:47 AM
> To: nanog@nanog.org 
> Subject: Re: OVH datacenter SBG2 in Strasbourg on fire 
>
> After sending them abuse reports for years with only an increase in 
malicious traffic, I have no expectation of anything they do getting 
better or being for the benefit of the internet as a > whole.  Only 
reason this is probably getting any attention from them is in hopes 
they don’t irreparably damage their IPO; they seem to have no issues 
with their customers' compromised servers damaging the businesses of 
others on a continuous basis.


Based on previous outages, I wouldn't agree. 
http://status.ovh.net/?do=details=15162#comment18119 

I will agree about the spam issues, but Octave Klaba has usually been 
pretty honest comparatively with what I'm used to from typical US 
based ILECs on outages.


Sincerely,

Eric Tykwinski
TrueNet, Inc.
P: 610-429-8300 

RE: OVH datacenter SBG2 in Strasbourg on fire ????

[keith]

That's the first mention of abuse I've seen. I've got huge blocks of OVH space 
blocked.. Lots of bad things coming from there. Not sure the fire will change 
things down the road as the sources are from all over the roadmap.

Sent from MailDroid

-Original Message-
From: eric-l...@truenet.com
To: nanog@nanog.org
Sent: Fri, 12 Mar 2021 9:54
Subject: RE: OVH datacenter SBG2 in Strasbourg on fire 

> -Original Message-
> From: NANOG  On Behalf Of 
> David Hubbard
> Sent: Friday, March 12, 2021 9:47 AM
> To: nanog@nanog.org
> Subject: Re: OVH datacenter SBG2 in Strasbourg on fire 
>
> After sending them abuse reports for years with only an increase in malicious 
> traffic, I have no expectation of anything they do getting better or being 
> for the benefit of the internet as a > whole.  Only reason this is probably 
> getting any attention from them is in hopes they don’t irreparably damage 
> their IPO; they seem to have no issues with their customers' compromised 
> servers damaging the businesses of others on a continuous basis.  

Based on previous outages, I wouldn't agree. 
http://status.ovh.net/?do=details=15162#comment18119
I will agree about the spam issues, but Octave Klaba has usually been pretty 
honest comparatively with what I'm used to from typical US based ILECs on 
outages.

Sincerely,

Eric Tykwinski
TrueNet, Inc.
P: 610-429-8300

RE: OVH datacenter SBG2 in Strasbourg on fire ????

[eric-list]

> -Original Message-
> From: NANOG  On Behalf Of 
> David Hubbard
> Sent: Friday, March 12, 2021 9:47 AM
> To: nanog@nanog.org
> Subject: Re: OVH datacenter SBG2 in Strasbourg on fire 
>
> After sending them abuse reports for years with only an increase in malicious 
> traffic, I have no expectation of anything they do getting better or being 
> for the benefit of the internet as a > whole.  Only reason this is probably 
> getting any attention from them is in hopes they don’t irreparably damage 
> their IPO; they seem to have no issues with their customers' compromised 
> servers damaging the businesses of others on a continuous basis.  

Based on previous outages, I wouldn't agree. 
http://status.ovh.net/?do=details=15162#comment18119
I will agree about the spam issues, but Octave Klaba has usually been pretty 
honest comparatively with what I'm used to from typical US based ILECs on 
outages.

Sincerely,

Eric Tykwinski
TrueNet, Inc.
P: 610-429-8300

NOAA/NWS data center water pipe burst

[Sean Donelan]

At 10:30 a.m. on March 9, 2021, the National Weather Service (NWS)
Headquarters building located in Silver Spring, Maryland, known as Silver
Spring Metro Complex 2 (SSMC-2), experienced a ruptured water pipe, which
caused significant and widespread flooding on the 7th floor and below. NWS
maintains a data center on one of the affected floors. When flooding was
discovered, power to the data center was turned off to protect the 
equipment and personnel who were onsite.

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[David Hubbard]

After sending them abuse reports for years with only an increase in malicious 
traffic, I have no expectation of anything they do getting better or being for 
the benefit of the internet as a whole.  Only reason this is probably getting 
any attention from them is in hopes they don’t irreparably damage their IPO; 
they seem to have no issues with their customers' compromised servers damaging 
the businesses of others on a continuous basis.  



On 3/12/21, 7:25 AM, "NANOG on behalf of Daniel Karrenberg" < > wrote:



On 11 Mar 2021, at 21:43, Randy Bush wrote:

> ...  but in a week or two
> i hope he can tell us results of more analysis. …

Actually just *the way* in which OVH communicates about this gives hope 
that we will indeed hear a useful analysis. It may be fortunate that 
this happened before they went public and thus corporate communications 
were ‘professionalised’ and vetted by the legal department. And yes 
he looked tired! Still did it in two languages. Good man.

Daniel

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Daniel Karrenberg]

On 11 Mar 2021, at 21:43, Randy Bush wrote:


...  but in a week or two
i hope he can tell us results of more analysis. …


Actually just *the way* in which OVH communicates about this gives hope 
that we will indeed hear a useful analysis. It may be fortunate that 
this happened before they went public and thus corporate communications 
were ‘professionalised’ and vetted by the legal department. And yes 
he looked tired! Still did it in two languages. Good man.


Daniel

Any Godaddy DNS / Firewall Admins here?

[DurgaPrasad - DatasoftComnet via NANOG]

Hello all,
Are there any Godaddy DNS / Firewall Admins here?
We are getting no response to our DNS servers 123.108.200.180,
123.108.201.51 from Godaddy DNS servers randomly for their DNS hosted
domains.
We use root.hints that are updated.
We have tried with powerdns recursor, unbound and also named with same
result.
We have checked we are not openrelay/We don't respond to IPs outside our
customer pools.

Can anyone help urgently.

Regards
DP


-- 
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Re: DOD prefixes and AS8003 / GRSCORP

[Tom Beecher]

>
> However, it appears that a Certificate of Good Standing is not required or
> ARIN didn't validate it in this case.
>

You don't know what ARIN did or did not do, or really anything about the
circumstances surrounding this other than what is gleanable from
public records. It's not a good look to chuck rocks at them like this.

Mr. Curran has helpfully provided the link to report suspected fraud.
That's the best place to take this discussion.

On Fri, Mar 12, 2021 at 6:36 AM Siyuan Miao  wrote:

> Hi John,
>
> My biggest concern is why the AS8003 was assigned to the company (GLOBAL
> RESOURCE SYSTEMS, LLC) even before its existence.
>
> When we were requesting resources or transfers, ARIN always asked us to
> provide a Certificate of Good Standing and we had to pay the state to order
> it.
>
> However, it appears that a Certificate of Good Standing is not required or
> ARIN didn't validate it in this case.
>
> Regards,
> Siyuan
>
> On Fri, Mar 12, 2021 at 7:17 PM John Curran  wrote:
>
>> On 11 Mar 2021, at 7:56 AM, Siyuan Miao  wrote:
>>
>>
>> Hi Folks,
>>
>> Just noticed that almost all DOD prefixes (
>> 7.0.0.0/8,11.0.0.0/8,22.0.0.0/8 and bunch of /22s)  are now announced
>> under AS8003 (GRSCORP) which was just formed a few months ago.
>>
>> It looks so suspicious. Does anyone know if it's authorized?
>>
>>
>> Siyuan -
>>
>> If you have concerns, you can confirm whether these IP address blocks are
>> being routed as intended by verification with their listed technical
>> contacts - e.g. https://search.arin.net/rdap/?query=22.0.0.0
>>
>> As I noted on this list several weeks back - "lack of routing history is
>> not at all a reliable indicator of the potential for valid routing of a
>> given IPv4 block in the future, so best practice suggest that allocated
>> address space should not be blocked by others without specific cause. Doing
>> otherwise opens one up to unexpected surprises when issued space suddenly
>> becomes more active in routing and is yet is inexplicably unreachable for
>> some destinations."
>>
>> Thanks!
>> /John
>>
>> John Curran
>> President and CEO
>> American Registry for Internet Numbers
>>
>>

Re: DOD prefixes and AS8003 / GRSCORP

[Siyuan Miao]

Hi Nick,

M0601699 was closed in 2006 according to Sunbiz (FL's official website):

http://search.sunbiz.org/Inquiry/CorporationSearch/SearchResultDetail?inquirytype=EntityName=Initial=GLOBALRESOURCESYSTEMS%20M06016990=forl-m0601699-a8147ffb-e7b4-41e1-a981-2bd8900de732=GLOBAL%20RESOURCE%20SYSTEMS%2C%20LLC=GLOBALRESOURCESYSTEMS%20M06016990

The new GLOBAL RESOURCE SYSTEMS, LLC (M2009226) was registered
on 10/13/2020.

http://search.sunbiz.org/Inquiry/CorporationSearch/SearchResultDetail?inquirytype=EntityName=Initial=GLOBALRESOURCESYSTEMS%20M20092260=forl-m2009226-80a9eec9-7fe2-4426-b3cd-9ebaa3e4e3b6=GLOBAL%20RESOURCE%20SYSTEMS%2C%20LLC=GLOBALRESOURCESYSTEMS%20M06016990

On Fri, Mar 12, 2021 at 7:52 PM Nick Hilliard  wrote:

> Siyuan Miao wrote on 12/03/2021 11:34:
> > My biggest concern is why the AS8003 was assigned to the company (GLOBAL
> > RESOURCE SYSTEMS, LLC) even before its existence.
>
> GRS LLC seems to have been around since 2006.
>
> > https://opencorporates.com/companies/us_fl/M0601699
>
> AS8003 was registered to them in Sep 2020:
>
> > ASNumber:   8003
> > ASName: GRS-DOD
> > ASHandle:   AS8003
> > RegDate:2020-09-14
> > Updated:2020-09-14
> > Ref:https://rdap.arin.net/registry/autnum/8003
>
> No doubt there is more information about the history of 8003 in WhoWas.
>
> Nick
>

Re: DOD prefixes and AS8003 / GRSCORP

[Nick Hilliard]

Siyuan Miao wrote on 12/03/2021 11:34:
My biggest concern is why the AS8003 was assigned to the company (GLOBAL 
RESOURCE SYSTEMS, LLC) even before its existence.


GRS LLC seems to have been around since 2006.


https://opencorporates.com/companies/us_fl/M0601699


AS8003 was registered to them in Sep 2020:


ASNumber:   8003
ASName: GRS-DOD
ASHandle:   AS8003
RegDate:2020-09-14
Updated:2020-09-14
Ref:https://rdap.arin.net/registry/autnum/8003


No doubt there is more information about the history of 8003 in WhoWas.

Nick

Re: DOD prefixes and AS8003 / GRSCORP

[John Curran]

On 12 Mar 2021, at 6:34 AM, Siyuan Miao 
mailto:avel...@misaka.io>> wrote:

Hi John,

My biggest concern is why the AS8003 was assigned to the company (GLOBAL 
RESOURCE SYSTEMS, LLC) even before its existence.

When we were requesting resources or transfers, ARIN always asked us to provide 
a Certificate of Good Standing and we had to pay the state to order it.

However, it appears that a Certificate of Good Standing is not required or ARIN 
didn't validate it in this case.

Siyuan -

If you believe that number resources may have been fraudulently obtained from 
ARIN, you can report the potential issue here -
https://www.arin.net/reference/tools/fraud_report/

Thanks!
/John

John Curran
President and CEO
American Registry for Internet Numbers

Re: DOD prefixes and AS8003 / GRSCORP

[Siyuan Miao]

Hi John,

My biggest concern is why the AS8003 was assigned to the company (GLOBAL
RESOURCE SYSTEMS, LLC) even before its existence.

When we were requesting resources or transfers, ARIN always asked us to
provide a Certificate of Good Standing and we had to pay the state to order
it.

However, it appears that a Certificate of Good Standing is not required or
ARIN didn't validate it in this case.

Regards,
Siyuan

On Fri, Mar 12, 2021 at 7:17 PM John Curran  wrote:

> On 11 Mar 2021, at 7:56 AM, Siyuan Miao  wrote:
>
>
> Hi Folks,
>
> Just noticed that almost all DOD prefixes (7.0.0.0/8,11.0.0.0/8,22.0.0.0/8
> and bunch of /22s)  are now announced under AS8003 (GRSCORP) which was just
> formed a few months ago.
>
> It looks so suspicious. Does anyone know if it's authorized?
>
>
> Siyuan -
>
> If you have concerns, you can confirm whether these IP address blocks are
> being routed as intended by verification with their listed technical
> contacts - e.g. https://search.arin.net/rdap/?query=22.0.0.0
>
> As I noted on this list several weeks back - "lack of routing history is
> not at all a reliable indicator of the potential for valid routing of a
> given IPv4 block in the future, so best practice suggest that allocated
> address space should not be blocked by others without specific cause. Doing
> otherwise opens one up to unexpected surprises when issued space suddenly
> becomes more active in routing and is yet is inexplicably unreachable for
> some destinations."
>
> Thanks!
> /John
>
> John Curran
> President and CEO
> American Registry for Internet Numbers
>
>

Re: DOD prefixes and AS8003 / GRSCORP

[John Curran]

On 11 Mar 2021, at 7:56 AM, Siyuan Miao 
mailto:avel...@misaka.io>> wrote:

Hi Folks,

Just noticed that almost all DOD prefixes 
(7.0.0.0/8,11.0.0.0/8,22.0.0.0/8 and 
bunch of /22s)  are now announced under AS8003 (GRSCORP) which was just formed 
a few months ago.

It looks so suspicious. Does anyone know if it's authorized?

Siyuan -

If you have concerns, you can confirm whether these IP address blocks are being 
routed as intended by verification with their listed technical contacts - e.g. 
https://search.arin.net/rdap/?query=22.0.0.0

As I noted on this list several weeks back - "lack of routing history is not at 
all a reliable indicator of the potential for valid routing of a given IPv4 
block in the future, so best practice suggest that allocated address space 
should not be blocked by others without specific cause. Doing otherwise opens 
one up to unexpected surprises when issued space suddenly becomes more active 
in routing and is yet is inexplicably unreachable for some destinations."

Thanks!
/John

John Curran
President and CEO
American Registry for Internet Numbers