At $WORK, we do Linux patching without breaking root mirrors or other such techniques.
We do extensive patch testing before rollout to production, so that by the time the patches get to production, it's a non-issue. We have 'development' and 'test' (staging) environments that are always patched first before production. Our clients aren't scared of patching because few incidents pop up in production - they're found during testing first. Full-scale upgrades (e.g., from RHEL4 to RHEL5) can have more fallout, but again, with proper testing in the dev/test areas, production incidents are infrequent, usually minor, and dealt with quickly. We *always* have a backout procedure - it's required to get any change through the formal Change Management process. Depending on the change, it may include restoring or rebuilding the server. Proper backups must also be taken in the case of OS upgrades, but only the normal data backups are done for simple patches. On some of the other OSes, breaking the root mirror used to be the standard for such upgrades. I haven't been involved in these upgrades for a few years, I don't know if this is still being done for Solaris and/or HP-UX. - Richard Bryce T. Pier wrote: > For years my employer has only patched *nix systems on an annual basis. > We've now been directed to apply security patches quarterly. Due to the > infrequency of patching in the past, there has developed a fairly high > level of paranoia around patching "breaking" things, particularly > related to servers not coming back from the post-patch reboot. To > mitigate these fears I've been asked to document procedures for > backing-out the applied patches and/or recovering the server in the > event of one not coming back up. > > Given that tools like RHN Satellite or Novell Zenworks don't have the > ability to do extensive pre-patch preparations like breaking hardware > root mirrors or running filesystem dumps, I have the impression that at > least in enterprise Linuxes there aren't frequently issues caused by > normal, regular patching activities. > > So I'm curious what other people are doing on the Linux platform. > > Do you use root disk mirrors and break the mirror prior to patching? > Do you utilize filesystem dumps (dumpe2fs, etc) or rely on enterprise > backups of the OS filesystems? > Do you use rpm rollbacks? > Rebuild / re-image the server if there are problems? > > Additionally, have you experienced many instances of patching tanking an > enterprise Linux server in the last couple of years? > > Thanks much! > _______________________________________________ Discuss mailing list [email protected] http://lopsa.org/cgi-bin/mailman/listinfo/discuss This list provided by the League of Professional System Administrators http://lopsa.org/
