Re: [uknof] Example of total DC loss

[Paul Brown]

Quoting Nick Hilliard :



from personal experience, I can say that the silence which ensues from a
power failure in a data centre is seriously creepy.



Oh yes - I was indirectly the cause of that happening at one hosting  
location in the Midlands about ten years ago when inserting a Dell  
blade server into its chassis, somehow the power rails shorted, and  
dumped about 60A straight through some copper which wasn't man enough  
for it and which then proceeded to vaporise a chunk of the blade  
motherboard, and both multi-way connectors (Blade side and chassis  
side).


One of my guys who was working behind it ended up inhaling a load of  
incredibly magic smoke, and the DCs VESDA install proved its worth by  
SCRAMing the entire floor.


Yeah - took a lot of other customers out with it, and we were banned  
from installing Dell blades at that location ever again.


Not a day long outage - took about an hour to verify that the site was  
good to restart, and to start feeding power in again, but yeah - the  
eerie silence after the white noise of several thousand server fans  
and such t like.


I bought a hundred and fifty HP DL140G2's straight afterwards and  
started ripping out the Dell kit.


P.

Re: [uknof] Hosting Firewall Advice

[Paul Brown]

 Quoting Paul Bone :


HI All,

    

    

    

    

   We currently run hosting of various Windows and Linux virtual  
servers for our customers behind an HA pair of Sonicwall firewalls.


    

    

    

    

   These are coming EOL and starting to reach capacity limits so we  
are looking to replace them, but I?m considering options.


    

    

    

    

   Just wondering what peoples thoughts are on the merits of a  
shared hardware firewall (we are starting to hit overlapping IP  
issues) vs virtual appliances or even a virtual Linux installation  
per client.


    

    

    

    

   And if anyone wants to share what they currently do I?d be grateful!

    

    

    

    

   Thanks

Paul

    

    

    

    


I'd suggest looking at virtual for this sort of thing, given that even  
the big "hardware" firewall vendors are tending to virtualising on  
their own hardware at the moment.


"Network Function Virtualisation" (NFV) is apparently the phrase the  
cool kids are using. Looks kinda like the old Inkra VSA that Savvis  
used to resell to me


P.

Re: [uknof] Example of total DC loss

[Paul Brown]

I'm hunting for an examples of long duration data centre outages in  
the UK, from a day of downtime to total data centre loss (explosion or  
some other industrial accident).


Cable and Wireless Watford

http://www.telegraph.co.uk/news/uknews/2282427/Metal-thieves-to-blame-for-Sainsburys-website-blackout.html

IIRC they had several instances of this happening to that particular site.

Also, Manchester Guardian Tunnels (BT) fire

https://www.theguardian.com/uk/2004/mar/30/simonjeffery

Also, not a datacentre outage, but a critical infrastructure failure  
when the Colossus backbone imploded.


https://www.theregister.co.uk/2001/11/20/uk_hit_by_major_adsl/

Quite a few (Including IIRC a CapGemini site on behalf of the NHS) on  
Brakspear Way in Hemel were rendered inoperable/inaccessible after the  
Buncefield disaster.


http://www.theregister.co.uk/2005/12/12/oil_blast_northgate/

There was also the Telehouse incident in 1998 (I think - may have been  
'97) when somebody hit the Master Off switch in one of the suites  
rather than the Open Door one, and then lots of ISP's kit responded  
badly to the power surge when the breaker was reset. Brief total  
outage (IIRC it took out a large chunk of LINX infrastructure briefly)  
and substantial impact for weeks afterwards as fragile PSUs in  
installed kit either failed on restart, or suffered premature death -  
I know of one Ascend MAX that never came back to life.


Paul