On 01/08/11 18:54, Volker Kuhlmann wrote:
> On Mon 01 Aug 2011 18:35:06 NZST +1200, C. Falconer wrote:
>> See if you can spot what's wrong with it.
>>    /bin/rm -f $output0 2>&1 /dev/null
>
> Easy.
> 1) It'll delete /dev/null

Correct - one internet to Volker.   I should have asked for replies off 
list to give other people a chance :))

Did anyone notice it has to be running as root to delete /dev/null  ?
Sadly it was.

And ssh simply refuses incoming connections if /dev/null doesn't exist.

And this particular box is inside a DC in another city, only manageable 
by SSH (the ilo licence was deemed too spendy.)

In the end the local DC admin dropped by and did a quick mknod.

Interestingly, everything else was fine... mta, http server, virtual 
machines, other monitoring daemons, everything else kept working.
A reboot of the box might have fixed it too, because udev would have 
recreated /dev on boot.

Long-term mitigation against it happening again is a chunk in the 
periodic sanity checker script.

if [ ! -c /dev/null ];
then
         echo "/dev/null not found or not a character file, recreating"
         /bin/mv /dev/null /dev/null.junk
         /bin/mknod /dev/null c 1 3
         /bin/chmod 666 /dev/null
         /bin/sleep 5
         /sbin/service sshd restart
fi


Any questions please ask



-- 
Craig Falconer

_______________________________________________
Linux-users mailing list
[email protected]
http://lists.canterbury.ac.nz/mailman/listinfo/linux-users

Reply via email to