Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Rich Kulawiec]

On Fri, Mar 12, 2021 at 02:46:51PM +, David Hubbard wrote:
> After sending them abuse reports for years with only an increase in
> malicious traffic, I have no expectation of anything they do getting
> better or being for the benefit of the internet as a whole.

This is a shared experience. It's abundantly clear that OVH is either
acting in concert with the abusers, or they *are* the abusers.  It doesn't
really matter which, the operational outcome is the same in either case.

The best course of action is to remove them from your (generic you)
view of the Internet via whatever means are most expedient.

---rsk

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Alain Hebert]

    One hope that the IPO will bring more pressure to evaluate the 
gain/loss ratio of handling the abuses to the community satisfaction.


    ( As for the the fire, I have to feel bad for all the staff working 
endless hours to get their customers back online ).


-
Alain Hebertaheb...@pubnix.net
PubNIX Inc.
50 boul. St-Charles
P.O. Box 26770 Beaconsfield, Quebec H9W 6G7
Tel: 514-990-5911  http://www.pubnix.netFax: 514-990-9443

On 3/12/21 10:19 AM, ke...@contoocook.net wrote:
That's the first mention of abuse I've seen. I've got huge blocks of 
OVH space blocked.. Lots of bad things coming from there. Not sure the 
fire will change things down the road as the sources are from all over 
the roadmap.


Sent from MailDroid <https://goo.gl/ODgwBb>

-Original Message-
From: eric-l...@truenet.com
To: nanog@nanog.org
Sent: Fri, 12 Mar 2021 9:54
Subject: RE: OVH datacenter SBG2 in Strasbourg on fire 

> -Original Message-
> From: NANOG <mailto:truenet@nanog.org>> On Behalf Of David Hubbard

> Sent: Friday, March 12, 2021 9:47 AM
> To: nanog@nanog.org <mailto:nanog@nanog.org>
> Subject: Re: OVH datacenter SBG2 in Strasbourg on fire 
>
> After sending them abuse reports for years with only an increase in 
malicious traffic, I have no expectation of anything they do getting 
better or being for the benefit of the internet as a > whole.  Only 
reason this is probably getting any attention from them is in hopes 
they don’t irreparably damage their IPO; they seem to have no issues 
with their customers' compromised servers damaging the businesses of 
others on a continuous basis.


Based on previous outages, I wouldn't agree. 
http://status.ovh.net/?do=details=15162#comment18119 
<http://status.ovh.net/?do=details=15162#comment18119>
I will agree about the spam issues, but Octave Klaba has usually been 
pretty honest comparatively with what I'm used to from typical US 
based ILECs on outages.


Sincerely,

Eric Tykwinski
TrueNet, Inc.
P: 610-429-8300 

RE: OVH datacenter SBG2 in Strasbourg on fire ????

[keith]

That's the first mention of abuse I've seen. I've got huge blocks of OVH space 
blocked.. Lots of bad things coming from there. Not sure the fire will change 
things down the road as the sources are from all over the roadmap.

Sent from MailDroid

-Original Message-
From: eric-l...@truenet.com
To: nanog@nanog.org
Sent: Fri, 12 Mar 2021 9:54
Subject: RE: OVH datacenter SBG2 in Strasbourg on fire 

> -Original Message-
> From: NANOG  On Behalf Of 
> David Hubbard
> Sent: Friday, March 12, 2021 9:47 AM
> To: nanog@nanog.org
> Subject: Re: OVH datacenter SBG2 in Strasbourg on fire 
>
> After sending them abuse reports for years with only an increase in malicious 
> traffic, I have no expectation of anything they do getting better or being 
> for the benefit of the internet as a > whole.  Only reason this is probably 
> getting any attention from them is in hopes they don’t irreparably damage 
> their IPO; they seem to have no issues with their customers' compromised 
> servers damaging the businesses of others on a continuous basis.  

Based on previous outages, I wouldn't agree. 
http://status.ovh.net/?do=details=15162#comment18119
I will agree about the spam issues, but Octave Klaba has usually been pretty 
honest comparatively with what I'm used to from typical US based ILECs on 
outages.

Sincerely,

Eric Tykwinski
TrueNet, Inc.
P: 610-429-8300

RE: OVH datacenter SBG2 in Strasbourg on fire ????

[eric-list]

> -Original Message-
> From: NANOG  On Behalf Of 
> David Hubbard
> Sent: Friday, March 12, 2021 9:47 AM
> To: nanog@nanog.org
> Subject: Re: OVH datacenter SBG2 in Strasbourg on fire 
>
> After sending them abuse reports for years with only an increase in malicious 
> traffic, I have no expectation of anything they do getting better or being 
> for the benefit of the internet as a > whole.  Only reason this is probably 
> getting any attention from them is in hopes they don’t irreparably damage 
> their IPO; they seem to have no issues with their customers' compromised 
> servers damaging the businesses of others on a continuous basis.  

Based on previous outages, I wouldn't agree. 
http://status.ovh.net/?do=details=15162#comment18119
I will agree about the spam issues, but Octave Klaba has usually been pretty 
honest comparatively with what I'm used to from typical US based ILECs on 
outages.

Sincerely,

Eric Tykwinski
TrueNet, Inc.
P: 610-429-8300

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[David Hubbard]

After sending them abuse reports for years with only an increase in malicious 
traffic, I have no expectation of anything they do getting better or being for 
the benefit of the internet as a whole.  Only reason this is probably getting 
any attention from them is in hopes they don’t irreparably damage their IPO; 
they seem to have no issues with their customers' compromised servers damaging 
the businesses of others on a continuous basis.  



On 3/12/21, 7:25 AM, "NANOG on behalf of Daniel Karrenberg" < > wrote:



On 11 Mar 2021, at 21:43, Randy Bush wrote:

> ...  but in a week or two
> i hope he can tell us results of more analysis. …

Actually just *the way* in which OVH communicates about this gives hope 
that we will indeed hear a useful analysis. It may be fortunate that 
this happened before they went public and thus corporate communications 
were ‘professionalised’ and vetted by the legal department. And yes 
he looked tired! Still did it in two languages. Good man.

Daniel

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Daniel Karrenberg]

On 11 Mar 2021, at 21:43, Randy Bush wrote:


...  but in a week or two
i hope he can tell us results of more analysis. …


Actually just *the way* in which OVH communicates about this gives hope 
that we will indeed hear a useful analysis. It may be fortunate that 
this happened before they went public and thus corporate communications 
were ‘professionalised’ and vetted by the legal department. And yes 
he looked tired! Still did it in two languages. Good man.


Daniel

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Randy Bush]

>>> It surprises that important sites don't do mirroring.
>> depends on what you mean by 'mirroring.' think latency.
> Though a best effort to mirror would be acceptable.  Maybe not up to
> the minute but at least a recovery.

depends on if the writer has to wait for it to hit the spinning oxide
1,000km away.  as i said, depends on what you mean by 'mirroring.'  i
would not recommend raid, drbd, ...  maybe periodic rsync, or an app
level sync designed for latency.

randy

---
ra...@psg.com
`gpg --locate-external-keys --auto-key-locate wkd ra...@psg.com`
signatures are back, thanks to dmarc header butchery

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Chris Cariffe]

On Thu, Mar 11, 2021 at 7:40 PM Randy Bush  wrote:

> It surprises that important sites don't do mirroring.
>
> depends on what you mean by 'mirroring.' think latency.
>
> randy
> --
>
> Though a best effort to mirror would be acceptable.  Maybe not up to the
> minute but at least a recovery.
>

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Randy Bush]

> It surprises that important sites don't do mirroring.

depends on what you mean by 'mirroring.'  think latency.

randy

---
ra...@psg.com
`gpg --locate-external-keys --auto-key-locate wkd ra...@psg.com`
signatures are back thanks to dmarc header mangling

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Rod Beck]

There are plenty of websites still down: https://www.centrepompidou.fr/en/.

It surprises that important sites don't do mirroring.

-R.


From: Johann 
Sent: Friday, March 12, 2021 12:41 AM
To: Rod Beck 
Cc: Randy Bush ; Lukas Tribus ; Eilers, Laura via 
NANOG 
Subject: Re: OVH datacenter SBG2 in Strasbourg on fire 

Several government websites have been impacted by fire. But >90% was 
operationnel during the same day after the incident.
There are DRP (seems to be working) and backups in other datacenters. There 
must remain exceptions or minor things broken.
But I think it's a pretty good result.

Johann

Le ven. 12 mars 2021 à 00:24, Rod Beck 
mailto:rod.b...@unitedcablecompany.com>> a 
écrit :
It is terrible timing with the company planning an IPO and also because the 
French government has heavily backed OVH. I think some government websites went 
down. France has a statist tradition that leads it to get involved in 
industries where has no expertise.




From: NANOG 
mailto:unitedcablecompany@nanog.org>>
 on behalf of Randy Bush mailto:ra...@psg.com>>
Sent: Thursday, March 11, 2021 9:43 PM
To: Lukas Tribus mailto:lu...@ltri.eu>>
Cc: Eilers, Laura via NANOG mailto:nanog@nanog.org>>
Subject: Re: OVH datacenter SBG2 in Strasbourg on fire 

>> Statement (in French) from Octave Klaba, containing some discussion
>> of the development of the fire (starts at ~ 4:30):
>> https://www.ovh.com/fr/images/sbg/index-fr.html
> English:
> https://www.ovh.com/fr/images/sbg/index-en.html

and a few hundred of us hoping we never have to stand in front of that
camera to explain a similar incident

a few things stood out: where backups are located, the ability to build
a lot of new servers and the supporting infra, ...  but in a week or two
i hope he can tell us results of more analysis.

randy

---
ra...@psg.com<mailto:ra...@psg.com>
`gpg --locate-external-keys --auto-key-locate wkd 
ra...@psg.com<mailto:ra...@psg.com>`
signatures are back, thanks to dmarc header butchery

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Johann]

Several government websites have been impacted by fire. But >90% was
operationnel during the same day after the incident.
There are DRP (seems to be working) and backups in other datacenters. There
must remain exceptions or minor things broken.
But I think it's a pretty good result.

Johann

Le ven. 12 mars 2021 à 00:24, Rod Beck  a
écrit :

> It is terrible timing with the company planning an IPO and also because
> the French government has heavily backed OVH. I think some government
> websites went down. France has a statist tradition that leads it to get
> involved in industries where has no expertise.
>
>
>
> --
> *From:* NANOG 
> on behalf of Randy Bush 
> *Sent:* Thursday, March 11, 2021 9:43 PM
> *To:* Lukas Tribus 
> *Cc:* Eilers, Laura via NANOG 
> *Subject:* Re: OVH datacenter SBG2 in Strasbourg on fire 
>
> >> Statement (in French) from Octave Klaba, containing some discussion
> >> of the development of the fire (starts at ~ 4:30):
> >> https://www.ovh.com/fr/images/sbg/index-fr.html
> > English:
> > https://www.ovh.com/fr/images/sbg/index-en.html
>
> and a few hundred of us hoping we never have to stand in front of that
> camera to explain a similar incident
>
> a few things stood out: where backups are located, the ability to build
> a lot of new servers and the supporting infra, ...  but in a week or two
> i hope he can tell us results of more analysis.
>
> randy
>
> ---
> ra...@psg.com
> `gpg --locate-external-keys --auto-key-locate wkd ra...@psg.com`
> signatures are back, thanks to dmarc header butchery
>

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Rod Beck]

It is terrible timing with the company planning an IPO and also because the 
French government has heavily backed OVH. I think some government websites went 
down. France has a statist tradition that leads it to get involved in 
industries where has no expertise.




From: NANOG  on behalf 
of Randy Bush 
Sent: Thursday, March 11, 2021 9:43 PM
To: Lukas Tribus 
Cc: Eilers, Laura via NANOG 
Subject: Re: OVH datacenter SBG2 in Strasbourg on fire 

>> Statement (in French) from Octave Klaba, containing some discussion
>> of the development of the fire (starts at ~ 4:30):
>> https://www.ovh.com/fr/images/sbg/index-fr.html
> English:
> https://www.ovh.com/fr/images/sbg/index-en.html

and a few hundred of us hoping we never have to stand in front of that
camera to explain a similar incident

a few things stood out: where backups are located, the ability to build
a lot of new servers and the supporting infra, ...  but in a week or two
i hope he can tell us results of more analysis.

randy

---
ra...@psg.com
`gpg --locate-external-keys --auto-key-locate wkd ra...@psg.com`
signatures are back, thanks to dmarc header butchery

Re: Re: OVH datacenter SBG2 in Strasbourg on fire 

[Sean Donelan]

In the 1990s, I spent long time researching and talking to people about 
the history of the old Automated Data Processing room code requirements. 
You can tell by the terminology "Automated Data Processing" the age of the 
original requirements. IBM helped write the original requirements in the 
1950s/1960s when one mainframe computer cost more than a building.


Times have changed.


On Thu, 11 Mar 2021, b...@theworld.com wrote:

I remember many years ago spec'ing a machine room at BU and coming to
loggerheads with the VP of building and grounds.

He (well, their rules) wanted low-temp sprinkler triggers, I wanted
the high-temp ones (I forget but I think 135F vs 175F.)

Me: I'll have over $2M in electrical equipment in that room!

Him: I have $20M in building surrounding your machine room, I win!

Re: Re: OVH datacenter SBG2 in Strasbourg on fire 

[bzs]

From: George Herbert 

...Interesting overview of fire damage.

I remember many years ago spec'ing a machine room at BU and coming to
loggerheads with the VP of building and grounds.

He (well, their rules) wanted low-temp sprinkler triggers, I wanted
the high-temp ones (I forget but I think 135F vs 175F.)

Me: I'll have over $2M in electrical equipment in that room!

Him: I have $20M in building surrounding your machine room, I win!


-- 
-Barry Shein

Software Tool & Die| b...@theworld.com | http://www.TheWorld.com
Purveyors to the Trade | Voice: +1 617-STD-WRLD   | 800-THE-WRLD
The World: Since 1989  | A Public Information Utility | *oo*

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Gavin Henry]

He looks sooo tired.

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Randy Bush]

>> Statement (in French) from Octave Klaba, containing some discussion
>> of the development of the fire (starts at ~ 4:30):
>> https://www.ovh.com/fr/images/sbg/index-fr.html
> English:
> https://www.ovh.com/fr/images/sbg/index-en.html

and a few hundred of us hoping we never have to stand in front of that
camera to explain a similar incident

a few things stood out: where backups are located, the ability to build
a lot of new servers and the supporting infra, ...  but in a week or two
i hope he can tell us results of more analysis.

randy

---
ra...@psg.com
`gpg --locate-external-keys --auto-key-locate wkd ra...@psg.com`
signatures are back, thanks to dmarc header butchery

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Lukas Tribus]

On Thu, 11 Mar 2021 at 18:10, Enno Rey  wrote:
>
> Statement (in French) from Octave Klaba, containing some discussion of the 
> development of the fire (starts at ~ 4:30):
>
> https://www.ovh.com/fr/images/sbg/index-fr.html

English:
https://www.ovh.com/fr/images/sbg/index-en.html

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Andy Ringsmuth]

> On Mar 11, 2021, at 10:41 AM, Sean Donelan  wrote:
> 
> If you pump 100s of megawatts of energy into a confined space (i.e. a large 
> data center), and there is an uncontrolled release of that energy (i.e. a 
> fire), that's a lot of energy which is going to go somewhere.
> Stored energy, batteries, backup fuel for generators, server plastic, etc.
> There is a lot of stuff that can burn in a data center, especially if you 
> start over-stuffing it.
> 
> China has the top 7 data centers over 100 megawatts.
> 
> http://worldstopdatacenters.com/power/
> 
> Ok, an ISP may not have a mega datacenter.  Even "small" data centers have a 
> large fuel load. They are major industrial operations.  Like other major 
> industrial fires, often the firefighter response is to rescue any trapped 
> people and then contain the fire until it burns itself out.
> 
> The "cloud" is just someone else's physical data center.

Obligatory:

https://xkcd.com/908/




Andy Ringsmuth
5609 Harding Drive
Lincoln, NE 68521-5831
(402) 304-0083
a...@andyring.com

“Better even die free, than to live slaves.” - Frederick Douglas, 1863

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Chris Boyd]

> On Mar 11, 2021, at 5:06 AM, Matt Harris  wrote:
> 
> There are plenty of effective options besides environmentally-destructive 
> Halon, dangerous-to-equipment water sprinkler, or dangerous-to-personnel CO2 
> for fire suppression these days. Some of the most common today are foam 
> systems like FM-200 or 3m's Novec. 

Novec and Solvay’s Galden are not really that much better than Halon. I guess 
it come down to which halogen do you want to release? Chlorine or Ffuorine?

https://www.engineeredfluids.com/post/are-pfas-the-next-pcbs

—Chris

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Sean Donelan]

If you pump 100s of megawatts of energy into a confined space (i.e. a 
large data center), and there is an uncontrolled release of that energy 
(i.e. a fire), that's a lot of energy which is going to go somewhere.

Stored energy, batteries, backup fuel for generators, server plastic, etc.
There is a lot of stuff that can burn in a data center, especially if you 
start over-stuffing it.


China has the top 7 data centers over 100 megawatts.

http://worldstopdatacenters.com/power/

Ok, an ISP may not have a mega datacenter.  Even "small" data centers have 
a large fuel load. They are major industrial operations.  Like other major 
industrial fires, often the firefighter response is to rescue any trapped 
people and then contain the fire until it burns itself out.


The "cloud" is just someone else's physical data center.


1975 AT New York City fire, documentary how it recovered:
https://www.youtube.com/watch?v=f_AWAmGi-g8

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Andy Ringsmuth]

> 
> https://us.ovhcloud.com/about/company/green-tech
> 
> Further I conjecture that a halon system would not be feasible in such a 
> structure.
> 
> Daniel
> 
> There are plenty of effective options besides environmentally-destructive 
> Halon, dangerous-to-equipment water sprinkler, or dangerous-to-personnel CO2 
> for fire suppression these days. Some of the most common today are foam 
> systems like FM-200 or 3m's Novec. 
> 
> - Matt
> 

I suppose I could be wrong, but I would imagine any of the various fire 
prevention/mitigation systems are less environmentally damaging than an entire 
building burning to the ground.


-Andy

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Christopher Morrow]

On Thu, Mar 11, 2021 at 10:45 AM Dale W. Carder  wrote:
> Sadly, the foams are not without environmental impact either with
> respect to PFAS.

what's the trade off on environmental impact for:
  "foam fire retardant" (or halon or ...)

vs:
  "Burning plastics and other carcinogen like materials"
  (or just blowing co2 into the space)

yes, halon isn't used and was 'not great' for hoomuns.
yes, co2 envelopment is unlikely to go well for hoomuns either.
yes, of course greenhouse gas release isn't great for gaia...

I'm wondering if the tradeoff is accounted for though :)

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Dale W. Carder]

Thus spake Matt Harris (m...@netfire.net) on Thu, Mar 11, 2021 at 05:06:46AM 
-0600:
> 
> There are plenty of effective options besides environmentally-destructive
> Halon, dangerous-to-equipment water sprinkler, or dangerous-to-personnel
> CO2 for fire suppression these days. Some of the most common today are foam
> systems like FM-200 or 3m's Novec.

Sadly, the foams are not without environmental impact either with
respect to PFAS.

Dale

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Sander Steffann]

> Again: all conjecture, which seems to be tolerated here. ;-)

It's all good food for thoughts! It's important to learn from these
things, because I (and I expect many others) assumed that fire
suppression systems would prevent something like this from happening.
It is good to think and talk about the limits of such systems.

Cheers!
Sander



signature.asc
Description: This is a digitally signed message part

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Daniel Karrenberg]

On 11 Mar 2021, at 11:46, Daniel Karrenberg wrote:

… Further I conjecture that a halon system would not be feasible in 
such a structure. …


Several of you have pointed out to me at varying level of politeness 
that ‘halon’ is no longer used. Sometimes I cannot hep but reveal my 
age … ;-)


I meant to say ‘Further I conjecture that the building could not have 
had a fire suppression system that requires reasonably airtight 
spaces.’. The between-the-lines there being that it may have been the 
very design of the building that lead to the rapid spreading of the 
fire.


Again: all conjecture, which seems to be tolerated here. ;-)

Daniel

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Matt Harris]

Matt Harris|Infrastructure Lead
816-256-5446|Direct
Looking for something?
Helpdesk Portal|Email Support|Billing Portal
We build and deliver end-to-end IT solutions.
On Thu, Mar 11, 2021 at 4:46 AM Daniel Karrenberg  wrote:

> Maybe the innovative ‘green’ design had something to do with the
> rapid spread of the fire and the way fire suppression was engineered:
>
> https://us.ovhcloud.com/about/company/green-tech
>
> Further I conjecture that a halon system would not be feasible in such a
> structure.
>
> Daniel
>

There are plenty of effective options besides environmentally-destructive
Halon, dangerous-to-equipment water sprinkler, or dangerous-to-personnel
CO2 for fire suppression these days. Some of the most common today are foam
systems like FM-200 or 3m's Novec.

- Matt

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Daniel Karrenberg]

On 10 Mar 2021, at 16:42, Andy Ringsmuth wrote:

Sad to see of course, but also a little surprising that fire 
suppression systems didn’t, well, suppress the fire.


Maybe the innovative ‘green’ design had something to do with the 
rapid spread of the fire and the way fire suppression was engineered:


https://us.ovhcloud.com/about/company/green-tech

Further I conjecture that a halon system would not be feasible in such a 
structure.


Daniel

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Rod Beck]

It is amazing that they can burn so well. Would never have imagined it.

Regards,

Roderick.


From: NANOG  on behalf 
of George Herbert 
Sent: Thursday, March 11, 2021 11:37 AM
To: Andy Ringsmuth 
Cc: nanog@nanog.org list 
Subject: Re: OVH datacenter SBG2 in Strasbourg on fire 

Sent from my iPhone

On Mar 10, 2021, at 7:45 AM, Andy Ringsmuth 
mailto:a...@andyring.com>> wrote:

Sad to see of course, but also a little surprising that fire suppression 
systems didn’t, well, suppress the fire.

Unless they didn’t exist?


I am assuming you haven’t had a real datacenter fire before.

I’ve had one fire, seen another, and had an accidental system firing of the gas 
system.

In the actual fire, caused by cooling system partial failure, there was no gas 
and it turned out the system to disable power on sprinkler discharge failed, 
then the power circuit breakers stayed live despite significant electrical 
short circuiting in the room as sprinklers fired, and then after 5 minutes the 
fire department arrived and sprayed water in for over 10 minutes without 
reducing arcing and fire before building power was successfully disabled and 
they could put the rest of the fire out.  This one could easily have destroyed 
its building, also the building Palo Alto Frys was in.

Fun fact: a rack of burning servers displaces sprinkler water around the rack, 
if it has a top on it, and even if not a vertical stack of burning servers 
pushes water down the front and back and slightly to the sides of burning 
servers, not through the systems themselves.

Fun fact: motherboards burn, as does chip encapsulation epoxy and all the 
wiring, the fan frames, board standoffs, essentially all the RAM and PCI board 
slots, some capacitors and surface mount devices... hard drives melt and the 
aluminum casings burn, SSD plastic casings generally burn.  GBICs and other 
laser diodes smell awful after they catch fire, though it can be hard to tell 
with everything else that smells.

Fun fact: It does not necessarily take many burning servers to put the room 
integrity at risk, even with sprinklers going and the fire department spraying 
water in.

Fun fact: All the servers that get wet but don’t burn will rust.  And 
everything in the room near the sprinklers that are going off will get wet.  
This is very sad as you watch many millions of dollars of brand new 42U racks 
prestuffed with HP enterprise servers oxidizing away while you wait for the 
garbage truck.

Fun fact: Combustion soot is conductive and even things that didn’t get wet 
probably are dead.

Fun fact: late era Sun Microsystems server boxes were very nice waxed 
cardboard, very well made, apparently fire resistant more than anyone would 
have thought but that were also water resistant enough that you’d have a new in 
box server submerged in a box still full to the brim with water days later.

Fun engineering advice: The window for critical data recovery from hard drives 
that are visibly corroding from water damage short of immersion is probably 
48-72 hours, but run them outside any casing on a fire resistant table and have 
all of CO2 and dry chemical fire extinguishers ready... and a fire hose, and 
handy building fire alarm.  Corroding water damaged SSDs are lower power draw 
and somewhat less likely to start another fire, but take the same precautions.


The fire I saw but that wasn’t mine burned a building with halon and pre action 
sprinklers more or less to the steel columns and roof beams, despite the fire 
department arriving in 5 minutes and trying to actively suppress.  Not enough 
openings for them to safely get to the fire before it was out of control, and 
not enough water flow available in all the sprinklers once it took off.

That one had evidently burned aluminum rack posts by the time it was over, not 
just melted them...


Systems and datacenters aren’t built to eliminate fire risk.  They just aren’t. 
 You can contain or control most office fires with sprinklers, and certainly 
evacuate.  Datacenters with emergency power batteries in the envelope often 
have enough stored energy to set the room on fire despite sprinklers.  If AC 
cutoffs fail and circuit breakers don’t trip mains power will as well.  Too 
many systems and storage and networking hardware components can burn.


-george

Re: OVH datacenter SBG2 in Strasbourg on fire 

[George Herbert]

Sent from my iPhone

> On Mar 10, 2021, at 7:45 AM, Andy Ringsmuth  wrote:
> 
> Sad to see of course, but also a little surprising that fire suppression 
> systems didn’t, well, suppress the fire.
> 
> Unless they didn’t exist?


I am assuming you haven’t had a real datacenter fire before.

I’ve had one fire, seen another, and had an accidental system firing of the gas 
system.  

In the actual fire, caused by cooling system partial failure, there was no gas 
and it turned out the system to disable power on sprinkler discharge failed, 
then the power circuit breakers stayed live despite significant electrical 
short circuiting in the room as sprinklers fired, and then after 5 minutes the 
fire department arrived and sprayed water in for over 10 minutes without 
reducing arcing and fire before building power was successfully disabled and 
they could put the rest of the fire out.  This one could easily have destroyed 
its building, also the building Palo Alto Frys was in.

Fun fact: a rack of burning servers displaces sprinkler water around the rack, 
if it has a top on it, and even if not a vertical stack of burning servers 
pushes water down the front and back and slightly to the sides of burning 
servers, not through the systems themselves.

Fun fact: motherboards burn, as does chip encapsulation epoxy and all the 
wiring, the fan frames, board standoffs, essentially all the RAM and PCI board 
slots, some capacitors and surface mount devices... hard drives melt and the 
aluminum casings burn, SSD plastic casings generally burn.  GBICs and other 
laser diodes smell awful after they catch fire, though it can be hard to tell 
with everything else that smells.

Fun fact: It does not necessarily take many burning servers to put the room 
integrity at risk, even with sprinklers going and the fire department spraying 
water in. 

Fun fact: All the servers that get wet but don’t burn will rust.  And 
everything in the room near the sprinklers that are going off will get wet.  
This is very sad as you watch many millions of dollars of brand new 42U racks 
prestuffed with HP enterprise servers oxidizing away while you wait for the 
garbage truck.

Fun fact: Combustion soot is conductive and even things that didn’t get wet 
probably are dead.

Fun fact: late era Sun Microsystems server boxes were very nice waxed 
cardboard, very well made, apparently fire resistant more than anyone would 
have thought but that were also water resistant enough that you’d have a new in 
box server submerged in a box still full to the brim with water days later.

Fun engineering advice: The window for critical data recovery from hard drives 
that are visibly corroding from water damage short of immersion is probably 
48-72 hours, but run them outside any casing on a fire resistant table and have 
all of CO2 and dry chemical fire extinguishers ready... and a fire hose, and 
handy building fire alarm.  Corroding water damaged SSDs are lower power draw 
and somewhat less likely to start another fire, but take the same precautions.


The fire I saw but that wasn’t mine burned a building with halon and pre action 
sprinklers more or less to the steel columns and roof beams, despite the fire 
department arriving in 5 minutes and trying to actively suppress.  Not enough 
openings for them to safely get to the fire before it was out of control, and 
not enough water flow available in all the sprinklers once it took off.

That one had evidently burned aluminum rack posts by the time it was over, not 
just melted them...


Systems and datacenters aren’t built to eliminate fire risk.  They just aren’t. 
 You can contain or control most office fires with sprinklers, and certainly 
evacuate.  Datacenters with emergency power batteries in the envelope often 
have enough stored energy to set the room on fire despite sprinklers.  If AC 
cutoffs fail and circuit breakers don’t trip mains power will as well.  Too 
many systems and storage and networking hardware components can burn.  


-george

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Sean Donelan]

Emergency responders run into buildings everyone else is running out of. 
Keep their safety in mind when designing your mega-data centers.



https://www.datacenterdynamics.com/en/news/fire-destroys-ovhclouds-sbg2-data-center-strasbourg/

The emergency services of Bas-Rhin sent 115 firefighters and 44 machines 
including six "cannon launchers" and two ladders, according to the DNA 
report. The facility is virtually on the French border, and other 
resources were mobilized across the border in Germany, according to a 
press release from Bas-Rhin. A Franco-German pump boat, Europa 1, carrying 
a German crew, helped put out the fire, taking water directly from the 
Rhine.

Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

[Jeff Shultz]

That'll work. :-)

On Wed, Mar 10, 2021 at 2:10 PM Sabri Berisha  wrote:

> - On Mar 10, 2021, at 1:41 PM, Jeff Shultz 
> wrote:
>
> The OVH datacenter is (was) in France. I bet you 10 bucks that the
>> fire department was on strike.
>
> Report I saw had the fire department on site in 3 minutes of the call.
> They even had a German-manned fireboat "Europa 1" working the fire from the
> water side.
>
> That's pretty impressive. It does make me wonder how long it took for them
> to be notified, and why on earth the fire spread so fast that the entire DC
> was lost...
>
> And because, for once, the French were not on strike, I donated $10 to the
> American Red Cross.
>
>
> Thanks,
>
> Sabri
>


-- 
Jeff Shultz

-- 
Like us on Social Media for News, Promotions, and other information!!

   
      
      
      














_ This message 
contains confidential information and is intended only for the individual 
named. If you are not the named addressee you should not disseminate, 
distribute or copy this e-mail. Please notify the sender immediately by 
e-mail if you have received this e-mail by mistake and delete this e-mail 
from your system. E-mail transmission cannot be guaranteed to be secure or 
error-free as information could be intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain viruses. The sender therefore does 
not accept liability for any errors or omissions in the contents of this 
message, which arise as a result of e-mail transmission. _

Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

[Randy Bush]

> No, French Superheroes flew in from Le Café du Peintre near the
> Bastille in under 30 nanoseconds. However, it was still futile.

jingoism does not deter fires

Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

[Töma Gavrichenkov]

Peace,

On Thu, Mar 11, 2021 at 1:10 AM Sabri Berisha  wrote:
> And because, for once, the French were not on strike, I donated $10 to the 
> American Red Cross.

I believe the American Red Cross has long given up even trying to
figure out who donates them how much and why.

--
Töma

Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

[Sabri Berisha]

- On Mar 10, 2021, at 1:41 PM, Jeff Shultz  wrote: 

>> The OVH datacenter is (was) in France. I bet you 10 bucks that the
>> fire department was on strike.
> Report I saw had the fire department on site in 3 minutes of the call. They 
> even
> had a German-manned fireboat "Europa 1" working the fire from the water side.

That's pretty impressive. It does make me wonder how long it took for them to 
be notified, and why on earth the fire spread so fast that the entire DC was 
lost... 

And because, for once, the French were not on strike, I donated $10 to the 
American Red Cross. 

Thanks, 

Sabri 

Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

[Rod Beck]

No, French Superheroes flew in from Le Café du Peintre near the Bastille in 
under 30 nanoseconds. However, it was still futile.

-R.


From: NANOG  on behalf 
of Jeff Shultz 
Sent: Wednesday, March 10, 2021 10:41 PM
To: North American Network Operators' Group 
Subject: Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

Report I saw had the fire department on site in 3 minutes of the call. They 
even had a German-manned fireboat "Europa 1" working the fire from the water 
side.

On Wed, Mar 10, 2021 at 11:37 AM Sabri Berisha 
mailto:sa...@cluecentral.net>> wrote:
- On Mar 10, 2021, at 11:28 AM, Bryan Holloway 
br...@shout.net<mailto:br...@shout.net> wrote:

Hi,

> Fire Department was there in under five minutes.

I assume your Enron DC was in the U.S.?

The OVH datacenter is (was) in France. I bet you 10 bucks that the
fire department was on strike.

Thanks,

Sabri


--
Jeff Shultz


Like us on Social Media for News, Promotions, and other information!!

  
[https://cdn0.iconfinder.com/data/icons/free-social-media-set/24/facebook-32.png]
 <https://www.facebook.com/SCTCWEB/>  
[https://cdn3.iconfinder.com/data/icons/follow-me/256/Instagram-32.png] 
<https://www.instagram.com/sctc_503/>  
[https://cdn2.iconfinder.com/data/icons/social-media-2142/192/Yelp-32.png] 
<https://www.yelp.com/biz/sctc-stayton-3>  
[https://cdn3.iconfinder.com/data/icons/free-social-icons/67/youtube_circle_color-32.png]
 <https://www.youtube.com/c/sctcvideos>







*** This message contains confidential information and is intended only for the 
individual named. If you are not the named addressee you should not 
disseminate, distribute or copy this e-mail. Please notify the sender 
immediately by e-mail if you have received this e-mail by mistake and delete 
this e-mail from your system. E-mail transmission cannot be guaranteed to be 
secure or error-free as information could be intercepted, corrupted, lost, 
destroyed, arrive late or incomplete, or contain viruses. The sender therefore 
does not accept liability for any errors or omissions in the contents of this 
message, which arise as a result of e-mail transmission. ***

Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

[Jeff Shultz]

Report I saw had the fire department on site in 3 minutes of the call. They
even had a German-manned fireboat "Europa 1" working the fire from the
water side.

On Wed, Mar 10, 2021 at 11:37 AM Sabri Berisha 
wrote:

> - On Mar 10, 2021, at 11:28 AM, Bryan Holloway br...@shout.net wrote:
>
> Hi,
>
> > Fire Department was there in under five minutes.
>
> I assume your Enron DC was in the U.S.?
>
> The OVH datacenter is (was) in France. I bet you 10 bucks that the
> fire department was on strike.
>
> Thanks,
>
> Sabri
>


-- 
Jeff Shultz

-- 
Like us on Social Media for News, Promotions, and other information!!

   
      
      
      














_ This message 
contains confidential information and is intended only for the individual 
named. If you are not the named addressee you should not disseminate, 
distribute or copy this e-mail. Please notify the sender immediately by 
e-mail if you have received this e-mail by mistake and delete this e-mail 
from your system. E-mail transmission cannot be guaranteed to be secure or 
error-free as information could be intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain viruses. The sender therefore does 
not accept liability for any errors or omissions in the contents of this 
message, which arise as a result of e-mail transmission. _

Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

[Rod Beck]

Sounds like France. 


From: NANOG  on behalf 
of Sabri Berisha 
Sent: Wednesday, March 10, 2021 8:34 PM
To: Bryan Holloway 
Cc: nanog 
Subject: Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

- On Mar 10, 2021, at 11:28 AM, Bryan Holloway br...@shout.net wrote:

Hi,

> Fire Department was there in under five minutes.

I assume your Enron DC was in the U.S.?

The OVH datacenter is (was) in France. I bet you 10 bucks that the
fire department was on strike.

Thanks,

Sabri

Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

[Sabri Berisha]

- On Mar 10, 2021, at 11:28 AM, Bryan Holloway br...@shout.net wrote:

Hi,

> Fire Department was there in under five minutes.

I assume your Enron DC was in the U.S.?

The OVH datacenter is (was) in France. I bet you 10 bucks that the
fire department was on strike.

Thanks,

Sabri

Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

[Bryan Holloway]

Vitrol™ -- Ask for it by name.

At $dayjob-1 I ran one of Enron's old (abandoned) datacenters which was 
built in the early aughts or earlier.


Even that had full pre-action systems, which we once triggered when one 
of my colleagues accidentally hooked up a battery backwards.


*poof*

Fire Department was there in under five minutes.

So yeah -- seems a little weird.


On 3/10/21 7:16 PM, Randy Bush wrote:

the conjecturbation is only surpassed by the vitrol

Re: AW: OVH datacenter SBG2 in Strasbourg on fire 

[Randy Bush]

the conjecturbation is only surpassed by the vitrol

AW: OVH datacenter SBG2 in Strasbourg on fire 

[Gunther Stammwitz]

According to these pictures a similar part of the datacenter (SBX4) was built 
with wood.
https://twitter.com/MedecineLibre/status/1369618937063817219
Ceilings at least. Walls?

-Ursprüngliche Nachricht-
Von: NANOG Im Auftrag von Andy Ringsmuth
Gesendet: Mittwoch, 10. März 2021 16:42
An: nanog@nanog.org
Betreff: Re: OVH datacenter SBG2 in Strasbourg on fire 


> On Mar 10, 2021, at 3:23 AM, Fredy Kuenzler  wrote:
>
> Very sad day for our colleagues at OVH AS16276 as they lost their 
> datacenter SBG-2 in Strasbourg/France completly („everything is 
> destroyed“) in a fire  and the neighboring SBG1/SBG3/SBG4 at least temporary.
>
> https://www.dna.fr/amp/faits-divers-justice/2021/03/10/strasbourg-impo
> rtant-incendie-dans-une-entreprise-situee-sur-un-site-seveso-au-port-d
> u-rhin

Sad to see of course, but also a little surprising that fire suppression 
systems didn’t, well, suppress the fire.

Unless they didn’t exist?



Andy Ringsmuth
5609 Harding Drive
Lincoln, NE 68521-5831
(402) 304-0083
a...@andyring.com

“Better even die free, than to live slaves.” - Frederick Douglas, 1863

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Marco Teixeira]

I'm sure this is just a coincidence:
"OVHcloud has launched the process for a public offering (IPO) on the Paris
stock exchange. The announcement came on Monday, two days before a fire
destroyed one of the company's 32 global data centers in Strasbourg."

Cheers,



On Wed, Mar 10, 2021 at 4:03 PM David Hubbard 
wrote:

> Was thinking the exact same thing; they and Digital Ocean seem to compete
> for the number two spot behind China as a malicious traffic source on all
> my public facing networks.
>
> From the pics, the place looked like it has had quite a few semi-trailer
> containers added around it, perhaps they were running a little too much
> equipment in the building and it required power / hvac augmenting but fire
> suppression was left as it was...
>
>
>
> On 3/10/21, 8:55 AM, "NANOG on behalf of JORDI PALET MARTINEZ via NANOG"
>  nanog@nanog.org> wrote:
>
> In addition to that, even if this is not good for many "honest" people
> that was using the DC, we need to take it in the positive side. In my own
> case, OVH is probably the cause of 80% of the abuse cases I report, and
> they never react. I'm convinced I'm not the only one, as I read in other
> ops mailing lists ...
>
> So, the positive side is a) during some days, we can see an
> interesting decrease in abuse cases, b) because the so many abuse cases,
> many OVH "honest" customers are often being filtered because they share
> addresses with the "bad guys", so it is an opportunity for them to move to
> alternative DCs that probably are more careful about "bad guys".
>
> A good topic for researchers :-)
>
> Regards,
> Jordi
> @jordipalet
>
>
>
> El 10/3/21 16:44, "NANOG en nombre de Andy Ringsmuth"
>  a...@andyring.com> escribió:
>
>
> > On Mar 10, 2021, at 3:23 AM, Fredy Kuenzler 
> wrote:
> >
> > Very sad day for our colleagues at OVH AS16276 as they lost
> their datacenter SBG-2 in Strasbourg/France completly („everything is
> destroyed“) in a fire  and the neighboring SBG1/SBG3/SBG4 at least
> temporary.
> >
> >
> https://www.dna.fr/amp/faits-divers-justice/2021/03/10/strasbourg-important-incendie-dans-une-entreprise-situee-sur-un-site-seveso-au-port-du-rhin
>
> Sad to see of course, but also a little surprising that fire
> suppression systems didn’t, well, suppress the fire.
>
> Unless they didn’t exist?
>
>
> 
> Andy Ringsmuth
> 5609 Harding Drive
> Lincoln, NE 68521-5831
> (402) 304-0083
> a...@andyring.com
>
> “Better even die free, than to live slaves.” - Frederick Douglas,
> 1863
>
>
>
>
> **
> IPv4 is over
> Are you ready for the new Internet ?
> http://www.theipv6company.com
> The IPv6 Company
>
> This electronic message contains information which may be privileged
> or confidential. The information is intended to be for the exclusive use of
> the individual(s) named above and further non-explicilty authorized
> disclosure, copying, distribution or use of the contents of this
> information, even if partially, including attached files, is strictly
> prohibited and will be considered a criminal offense. If you are not the
> intended recipient be aware that any disclosure, copying, distribution or
> use of the contents of this information, even if partially, including
> attached files, is strictly prohibited, will be considered a criminal
> offense, so you must reply to the original sender to inform about this
> communication and delete it.
>
>
>
>
>

Re: OVH datacenter SBG2 in Strasbourg on fire ????

[Rich Kulawiec]

If you give people the means to hurt you, and they do it, and
you take no action except to continue giving them the means to
hurt you, and they take no action except to keep hurting you,
then one of the ways you can describe the situation is "it isn't
scaling well".
--- Paul Vixie (on NANOG)

1. I have every OVH network block that I'm aware of permanently firewalled
out.  I recommend the same to everyone else, unless you're doing research.

2. When something burns that's not supposed to burn, that was designed
not to burn, that was built not to burn, that was operated not to burn,
then one of the distinct possibilities is that it wasn't an accident.

---rsk

Re: OVH datacenter SBG2 in Strasbourg on fire 

[David Hubbard]

Was thinking the exact same thing; they and Digital Ocean seem to compete for 
the number two spot behind China as a malicious traffic source on all my public 
facing networks.

From the pics, the place looked like it has had quite a few semi-trailer 
containers added around it, perhaps they were running a little too much 
equipment in the building and it required power / hvac augmenting but fire 
suppression was left as it was...



On 3/10/21, 8:55 AM, "NANOG on behalf of JORDI PALET MARTINEZ via NANOG" 
 wrote:

In addition to that, even if this is not good for many "honest" people that 
was using the DC, we need to take it in the positive side. In my own case, OVH 
is probably the cause of 80% of the abuse cases I report, and they never react. 
I'm convinced I'm not the only one, as I read in other ops mailing lists ...

So, the positive side is a) during some days, we can see an interesting 
decrease in abuse cases, b) because the so many abuse cases, many OVH "honest" 
customers are often being filtered because they share addresses with the "bad 
guys", so it is an opportunity for them to move to alternative DCs that 
probably are more careful about "bad guys".

A good topic for researchers :-)

Regards,
Jordi
@jordipalet



El 10/3/21 16:44, "NANOG en nombre de Andy Ringsmuth" 
 escribió:


> On Mar 10, 2021, at 3:23 AM, Fredy Kuenzler  
wrote:
> 
> Very sad day for our colleagues at OVH AS16276 as they lost their 
datacenter SBG-2 in Strasbourg/France completly („everything is destroyed“) in 
a fire  and the neighboring SBG1/SBG3/SBG4 at least temporary.
> 
> 
https://www.dna.fr/amp/faits-divers-justice/2021/03/10/strasbourg-important-incendie-dans-une-entreprise-situee-sur-un-site-seveso-au-port-du-rhin

Sad to see of course, but also a little surprising that fire 
suppression systems didn’t, well, suppress the fire.

Unless they didn’t exist?



Andy Ringsmuth
5609 Harding Drive
Lincoln, NE 68521-5831
(402) 304-0083
a...@andyring.com

“Better even die free, than to live slaves.” - Frederick Douglas, 1863




**
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or 
confidential. The information is intended to be for the exclusive use of the 
individual(s) named above and further non-explicilty authorized disclosure, 
copying, distribution or use of the contents of this information, even if 
partially, including attached files, is strictly prohibited and will be 
considered a criminal offense. If you are not the intended recipient be aware 
that any disclosure, copying, distribution or use of the contents of this 
information, even if partially, including attached files, is strictly 
prohibited, will be considered a criminal offense, so you must reply to the 
original sender to inform about this communication and delete it.

Re: OVH datacenter SBG2 in Strasbourg on fire 

[John Von Essen]

So your saying my “bot” dashboard should show a decrease in volume today? 
Interesting… I might run some stats today to see if there is a noticeable drop 
in Europe.

-John

> On Mar 10, 2021, at 10:53 AM, JORDI PALET MARTINEZ via NANOG 
>  wrote:
> 
> In addition to that, even if this is not good for many "honest" people that 
> was using the DC, we need to take it in the positive side. In my own case, 
> OVH is probably the cause of 80% of the abuse cases I report, and they never 
> react. I'm convinced I'm not the only one, as I read in other ops mailing 
> lists ...
> 
> So, the positive side is a) during some days, we can see an interesting 
> decrease in abuse cases, b) because the so many abuse cases, many OVH 
> "honest" customers are often being filtered because they share addresses with 
> the "bad guys", so it is an opportunity for them to move to alternative DCs 
> that probably are more careful about "bad guys".
> 
> A good topic for researchers :-)
> 
> Regards,
> Jordi
> @jordipalet
> 
> 
> 
> El 10/3/21 16:44, "NANOG en nombre de Andy Ringsmuth" 
>  a...@andyring.com> escribió:
> 
> 
>> On Mar 10, 2021, at 3:23 AM, Fredy Kuenzler  wrote:
>> 
>> Very sad day for our colleagues at OVH AS16276 as they lost their 
>> datacenter SBG-2 in Strasbourg/France completly („everything is destroyed“) 
>> in a fire  and the neighboring SBG1/SBG3/SBG4 at least temporary.
>> 
>> https://www.dna.fr/amp/faits-divers-justice/2021/03/10/strasbourg-important-incendie-dans-une-entreprise-situee-sur-un-site-seveso-au-port-du-rhin
> 
>Sad to see of course, but also a little surprising that fire suppression 
> systems didn’t, well, suppress the fire.
> 
>Unless they didn’t exist?
> 
> 
>
>Andy Ringsmuth
>5609 Harding Drive
>Lincoln, NE 68521-5831
>(402) 304-0083
>a...@andyring.com
> 
>“Better even die free, than to live slaves.” - Frederick Douglas, 1863
> 
> 
> 
> 
> **
> IPv4 is over
> Are you ready for the new Internet ?
> http://www.theipv6company.com
> The IPv6 Company
> 
> This electronic message contains information which may be privileged or 
> confidential. The information is intended to be for the exclusive use of the 
> individual(s) named above and further non-explicilty authorized disclosure, 
> copying, distribution or use of the contents of this information, even if 
> partially, including attached files, is strictly prohibited and will be 
> considered a criminal offense. If you are not the intended recipient be aware 
> that any disclosure, copying, distribution or use of the contents of this 
> information, even if partially, including attached files, is strictly 
> prohibited, will be considered a criminal offense, so you must reply to the 
> original sender to inform about this communication and delete it.
> 
> 
> 

Re: OVH datacenter SBG2 in Strasbourg on fire 

[JORDI PALET MARTINEZ via NANOG]

In addition to that, even if this is not good for many "honest" people that was 
using the DC, we need to take it in the positive side. In my own case, OVH is 
probably the cause of 80% of the abuse cases I report, and they never react. 
I'm convinced I'm not the only one, as I read in other ops mailing lists ...

So, the positive side is a) during some days, we can see an interesting 
decrease in abuse cases, b) because the so many abuse cases, many OVH "honest" 
customers are often being filtered because they share addresses with the "bad 
guys", so it is an opportunity for them to move to alternative DCs that 
probably are more careful about "bad guys".

A good topic for researchers :-)

Regards,
Jordi
@jordipalet
 
 

El 10/3/21 16:44, "NANOG en nombre de Andy Ringsmuth" 
 escribió:


> On Mar 10, 2021, at 3:23 AM, Fredy Kuenzler  wrote:
> 
> Very sad day for our colleagues at OVH AS16276 as they lost their 
datacenter SBG-2 in Strasbourg/France completly („everything is destroyed“) in 
a fire  and the neighboring SBG1/SBG3/SBG4 at least temporary.
> 
> 
https://www.dna.fr/amp/faits-divers-justice/2021/03/10/strasbourg-important-incendie-dans-une-entreprise-situee-sur-un-site-seveso-au-port-du-rhin

Sad to see of course, but also a little surprising that fire suppression 
systems didn’t, well, suppress the fire.

Unless they didn’t exist?



Andy Ringsmuth
5609 Harding Drive
Lincoln, NE 68521-5831
(402) 304-0083
a...@andyring.com

“Better even die free, than to live slaves.” - Frederick Douglas, 1863




**
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or 
confidential. The information is intended to be for the exclusive use of the 
individual(s) named above and further non-explicilty authorized disclosure, 
copying, distribution or use of the contents of this information, even if 
partially, including attached files, is strictly prohibited and will be 
considered a criminal offense. If you are not the intended recipient be aware 
that any disclosure, copying, distribution or use of the contents of this 
information, even if partially, including attached files, is strictly 
prohibited, will be considered a criminal offense, so you must reply to the 
original sender to inform about this communication and delete it.

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Matt Harris]

Matt Harris|Infrastructure Lead
816-256-5446|Direct
Looking for something?
Helpdesk Portal|Email Support|Billing Portal
We build and deliver end-to-end IT solutions.
On Wed, Mar 10, 2021 at 9:43 AM Andy Ringsmuth  wrote:

>
> > On Mar 10, 2021, at 3:23 AM, Fredy Kuenzler  wrote:
> >
> > Very sad day for our colleagues at OVH AS16276 as they lost their
> datacenter SBG-2 in Strasbourg/France completly („everything is destroyed“)
> in a fire  and the neighboring SBG1/SBG3/SBG4 at least temporary.
> >
> >
> https://www.dna.fr/amp/faits-divers-justice/2021/03/10/strasbourg-important-incendie-dans-une-entreprise-situee-sur-un-site-seveso-au-port-du-rhin
>
> Sad to see of course, but also a little surprising that fire suppression
> systems didn’t, well, suppress the fire.
>
> Unless they didn’t exist?
>

I was just discussing this with a buddy of mine. I'm hoping we get a
post-mortem that helps us understand what happened and how we can all do
better potentially as an industry. The fact that fire fighters (ostensibly
professionals, based out of what appears to be an industrial areas no less)
were unable to get the fire under control definitely indicates to me that
there was something exceptional going on there beyond a typical fire in a
data center environment.

There had to have been cascading failures somewhere along the line, though,
imho: whether that involve the engineering, implementation, or execution
stage or multiple thereof remains to be seen...

 - Matt

Re: OVH datacenter SBG2 in Strasbourg on fire 

[Andy Ringsmuth]

> On Mar 10, 2021, at 3:23 AM, Fredy Kuenzler  wrote:
> 
> Very sad day for our colleagues at OVH AS16276 as they lost their datacenter 
> SBG-2 in Strasbourg/France completly („everything is destroyed“) in a fire  
> and the neighboring SBG1/SBG3/SBG4 at least temporary.
> 
> https://www.dna.fr/amp/faits-divers-justice/2021/03/10/strasbourg-important-incendie-dans-une-entreprise-situee-sur-un-site-seveso-au-port-du-rhin

Sad to see of course, but also a little surprising that fire suppression 
systems didn’t, well, suppress the fire.

Unless they didn’t exist?



Andy Ringsmuth
5609 Harding Drive
Lincoln, NE 68521-5831
(402) 304-0083
a...@andyring.com

“Better even die free, than to live slaves.” - Frederick Douglas, 1863

OVH datacenter SBG2 in Strasbourg on fire 

[Fredy Kuenzler]

Very sad day for our colleagues at OVH AS16276 as they lost their datacenter 
SBG-2 in Strasbourg/France completly („everything is destroyed“) in a fire  
and the neighboring SBG1/SBG3/SBG4 at least temporary.

https://www.dna.fr/amp/faits-divers-justice/2021/03/10/strasbourg-important-incendie-dans-une-entreprise-situee-sur-un-site-seveso-au-port-du-rhin

--
Fredy Künzler

Init7 (Switzerland) Ltd.
Technoparkstrasse 5
CH-8406 Winterthur
https://www.init7.net/