Thought I'd share a Linux reated bit of panic caused by a raid array...

I've been banging my head against that silly file associations problem
(I might restore my whole home directory...) and I was having weird
problems with my data volume as well, which is NFS mounted from my
server out in the shed.

I assumed it was all tied up with the workstation issues, and kept
bashing away until I noticed that I had no home page on my internal
server.   Apache was up, but no data being served. Hmmmmm.

Open my email, and I had one of the dreaded mdadm notifications telling
me that I'd lost two drives out of my raid 5 array..... TWO drives!
ARGH!!!!  Sweaty palms, cold sweats....

So I go out to the shed, and hard power off the box, bring it back up,
sure enough, only 3 of the 5 drives appear.  Hmmmmmmm.  Both of the
failed drives are on the fancy add-on IDE controller.  Could be a
controller fault.

I piddle around with mdadm --re-add and --assemble a bit to see if it
was some sort of soft fault, and no joy.

OK, out with the drives, test them individually in another PC and figure
out what has gone wrong, praying that they're not really dead as I have
just over 800Gb in my scratch drive that wasn't backed up...  (Yeah,
yeah, I know...)

The drives appear to be OK in another machine.  Hmmmm.  Controller?

Oh....

Does it have to be plugged into the PCI slot all the way?

Right, plug it in.  Turn it all on again.  Hmmmm, still two drives down.

mdadm --re-add /dev/md1 /dev/hdc1 /dev/hdd1
mdadm -R /dev/md1

Hey-presto, it's happy....  I like linux software raid...  

The moral of the story?  When building your server, screw in the
fancy-schmancy 4 channel IDE card, otherwise it will fall out eventually
due to vibration in the server, and cause hair loss!!!!!

Hmmm, now back to my silly file association problems.

Cheers, Me.

Reply via email to