Ah, the downtime tales I can tell...

Back in my not-that-distant past working for a supercomputing center
downtime of the machine room was usually caused by one of two
problems: 1) lightning strikes taking down the power grid or 2)
electricians playing, "Hey this breaker isn't labeled!  Let's see
what's on it!"

#1 was a problem for a long time because a) there was no generator
behind the UPS and b) the bigger Crays didn't fit on the UPS anyway.
#2 was just... a problem.

Y2K (remember that to-do?) rolled around and generators behind the UPS
became a requirement.  The bigger Crays weren't an issue anymore.  The
generators were obtained.  The real humor of this is that the
auto-start for the generators wasn't installed for almost a year
afterwards - until then someone had to push a button to start them.

2.5 years later (& no site-wide downtime) a coworker & I were both
attending the a conference across the country. It was the middle of
the summer and while talking to coworkers one mentioned a huge mass of
thunderstorms rolling through. Not long after we both lost connection
to servers in the machine room.

We found out later that, yet again, a lightning strike had hit the
power grid, and, as always, the UPS swiftly took over.  After N
minutes the auto-start fired up the generators.   However there was
never any means installed to get the power from the generators to the
UPS & power system, so the site immediately went dark.

Did I mention the machine room was in the basement of a company that
designed & built nuclear power plants?

(I should save this story to get drinks outta it)

Moose
_______________________________________________
Discuss mailing list
[email protected]
http://lopsa.org/cgi-bin/mailman/listinfo/discuss
This list provided by the League of Professional System Administrators
 http://lopsa.org/

Reply via email to