I've done something like that....though this'll be the first time I've
admitted it.

A unit I support(ed) had a pair of v240s...for their main website.....alice
and betty....alice was the primary server and betty was the secondary.  But, I
had then upgraded the boxes to Solaris 10, and we changed to functional naming
scheme...so they had become <unit>-web1 and <unit>-web2....

Got a notification from vxvm that the rootmirror had failed in <unit>-web2, so
I went and found a new a drive and went to replace it.

When I got back to my desk there was a notification that the rootmirror had
also failed in <unit>-web1.  Wow, what luck that the that the rootmirror in a
pair of same age servers would die the same day!  Even though rootmirror in
alice was older....

But, then I ran into a snag....vxvm wasn't cooperating the way I expected with
adding disk after I had removed rootmirror for replacement.

Oh darn...that's right....to not have downtime when I upgraded the
servers....I had upgraded betty to Solaris 10 first, and named it <unit>-web1

I had in my mind alice, primary, <unit>-web1....betty, secondary,
<unit>-web2....I had replaced the wrong rootmirror.

After that I fixed the labels on the machines.

I didn't tell anybody that what had done...I kept the good rootmirror that I
had pulled at my desk as a future slide it into some other server that needs a
new disk, instead of find manager for key to supply room. (used to be the
closet where they locked up the paper for the mainframe printers...)

We used to keep our spares in the room next to the data center, but a director
came through and said what's all this junk here for...and had it all thrown
out to make additional office space. (office space that has never been
used....)  Just like the rearranged our space to go from 4 cubes to 6 cubes. 
With 2 small cubes for student admins and 1 big cube for another admin for our
group.  .... well, we use the admin cube for da110, and the student cube got
re-consolidated into one big cube....its vacant the moment....  Those cubes
are in room 9, our group is currently using room 11.  Originally 4 cubes, but
had gotten redone into 7 cubes.  We're only 6 (since May 2nd) before that
there were two vacant cubes....and we were advertising to positions :P  The
3rd position has been vacant so long (about 6 or 7 years now?  Google lured
him away....) that they aren't sure where the funding for it is anymore.  Of
course, it used to be the e-mail administrator position.... which is what I
spend a lot of my time doing (but having outsourced it...is supposed to free
us up from doing e-mail).

On 5/14/2011 1:40 PM, Ross Lonstein wrote:
> Not my mistake but a good one... I was a junior sysadmin when one of
> the disks in the FC array under an important Sybase DB failed. Go to
> the machine room, do my thing with vxdisk, vxedit and luxadm. Double
> check my notes before removing the drive. Hmmm, the last spare drive
> in the chassis has also failed. The Senior SA walks in, impatiently
> pushes me aside, asks which drive failed and before I can answer,
> waves me off and reaches over and pulls the *wrong drive*. The console
> is showing errors I haven't seen before. Over the fan noise and A/C, I
> can hear the Openview console playing the flushing toilet sound we
> configured for certain traps. Then my pager goes off. Senior SA looks
> at me with shock on his face, tells me to get started with restoring
> from tape and hurries out.

-- 
Who: Lawrence K. Chen, P.Eng. - W0LKC - Senior Unix Systems Administrator
For: Enterprise Server Technologies (EST) -- & SafeZone Ally
Snail: Computing and Telecommunications Services (CTS)
Kansas State University, 109 East Stadium, Manhattan, KS 66506-3102
Phone: (785) 532-4916 - Fax: (785) 532-3515 - Email: [email protected]
Web: http://www-personal.ksu.edu/~lkchen - Where: 11 Hale Library

_______________________________________________
Discuss mailing list
[email protected]
https://lists.lopsa.org/cgi-bin/mailman/listinfo/discuss
This list provided by the League of Professional System Administrators
 http://lopsa.org/

Reply via email to