If he hadn't been gracious enough to accept an I told you so from an employee who had been working for over 24 hours because of something he caused, I wouldn't want to be working there.
In hindsight, you did the right thing, because it allowed you to see who you were working for. On Wed, Feb 24, 2010 at 3:23 PM, Charlie Kaiser <[email protected]>wrote: > This was about 6 years ago. I took over as senior admin at a client that > does boat/rv loans. Our firm did their outsourced IT. They had a recent > implementation of Cisco Voip including Unity 2.4.something (voicemail). > > We were in the process of doing major app/infrastructure upgrades. I had > been pushing to get them off the standalone unity implementation. For those > who don't remember unity 2.4, it had, all on one box, windows 2000 AD, > Exchange 5.5, and Unity. That update was on the planning board... > > One day we had a power outage in the office around 10 AM. I frantically > shut > down all the servers as quickly as I could, since our UPSs were, shall we > say, less than optimal for the infrastructure. I got everything shut down > and was wiping the sweat off my brow when the CEO walked in and said "I > need > the voicemail server on so I can check my voicemail once more; I'm > expecting > a call". I said no. Too risky. Bad things could happen. You have to trust > me > on this. He said "I really need this. I understand; turn it back on NOW." > OK; you're the boss. I got it back up and running and just as he was > connecting back to his vm, the UPS battery died. > > Sigh. > > A couple of hours later the power came back on and I went through my > restart > checklist. When we got to the VM server, it came up "can't find NTLDR" or > something similar. Nice... The OS was hosed. None of my tricks worked, so > we > got MS on the line. After several hours with them, we finally got the OS > back somehow; don't recall all the details. Then we tried to start AD and > the DB was hosed. So several more hours with MS. Finally, about 6 pm, we > got > AD back. Then the exchange DB was toast. Of course, we'd just installed a > new backup server a few days previously and didn't have good backups yet. > So > we had to fix the DB. That took more hours. Finally, about 11:30, we had > the > MS stuff working. > > We then tried to launch Unity. No such luck. Called Cisco. "Oh; good thing > you called when you did; this product is EOL at midnight tonight". They > said > no prob though; they'd work with us as long as we needed since we opened > the > case before midnight. > Spent the next 10 hours on the phone with Cisco getting Unity back up. > > Finally had it all working again. By now it had been over 24 hours and I > was > a bit fried. My boss was in the conference room with the CEO when I walked > in. The CEO looked sheepishly at me; he knew what had happened and why. I > pointed my finger at him and said, probably a bit more forcefully than I > should have "The next time I tell you something's a bad idea, you'd better > pay attention. And when I tell you no, I really mean no". My boss hid a > shocked smile behind his laptop screen and the CEO just blushed and nodded. > > I left the building totally wiped out, ready to rebuild my resume once I > realized what I'd done. But instead of getting fired I got a nice bonus > for > putting everything back together. > > Probably used up ALL of my CEO luck with that one episode... :-) > > *********************** > Charlie Kaiser > [email protected] > Kingman, AZ > *********************** > > > -----Original Message----- > > From: John Aldrich [mailto:[email protected]] > > Sent: Wednesday, February 24, 2010 12:46 PM > > To: NT System Admin Issues > > Subject: RE: Stupid user tricks! > > > > Now THAT would be an interesting story! :-) > > > > > > > > > > -----Original Message----- > > From: Charlie Kaiser [mailto:[email protected]] > > Sent: Wednesday, February 24, 2010 2:42 PM > > To: NT System Admin Issues > > Subject: RE: Stupid user tricks! > > > > You mean 1100? Yeah, but I'm still shaking my head about the > > whole coke on top of the JBOD thing in the first place... ;-) > > > > How the trays all had liquid coke in them and the unit is > > still functional is beyond my comprehension. I'm just glad > > it's my boss there and not me. I tend to open my can of WA > > when that sort of thing happens. He's more diplomatic... > > > > Maybe sometime I'll tell the story of the time I chewed the > > CEO a new one and kept my job... <G> > > > > *********************** > > Charlie Kaiser > > [email protected] > > Kingman, AZ > > *********************** > > > > > -----Original Message----- > > > From: Kurt Buff [mailto:[email protected]] > > > Sent: Wednesday, February 24, 2010 12:34 PM > > > To: NT System Admin Issues > > > Subject: Re: Stupid user tricks! > > > > > > You can't count to 12? > > > > > > On Wed, Feb 24, 2010 at 10:52, Charlie Kaiser > > > <[email protected]> wrote: > > > > Speaking of SUT... Just got a call from my boss at one of > > > our clients. > > > > We've been tracking down the cause of broken disk caddies > > on a JBOD > > > > (all of the caddy handles were broken). Turns out someone > > > spilled coke > > > > on the unit and couldn't figure out how to release the > > > handles. Broke all 12... > > > > > > > > That's just wrong in more ways than I can count... > > > > > > > > *********************** > > > > Charlie Kaiser > > > > [email protected] > > > > Kingman, AZ > > > > *********************** > > > > > > > > > > > > ~ Finally, powerful endpoint security that ISN'T a resource > > > hog! ~ ~ > > > > <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ > > > > > > > > > > ~ Finally, powerful endpoint security that ISN'T a resource > > hog! ~ ~ > > > <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ > > > > > > > > > ~ Finally, powerful endpoint security that ISN'T a resource > > hog! ~ ~ > > <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ > > > > > > > > ~ Finally, powerful endpoint security that ISN'T a resource > > hog! ~ ~ > > <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ > > > > > ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ > ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ > > ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~
