(I'm resending because Google Groups destroyed my formatting) On Nov 18, 7:00 am, James Turnbull <[EMAIL PROTECTED]> wrote: > Ceri Storey wrote: > > I suppose what I'm asking, is firstly, is anyone else doing > > anything like this themselves? I imagine someone must be. If so, > > are they using wrapper scripts (as we do) or some other mechanism?
I'm doing precicely this for all our datacenters, and I don't think it misses the point. :) We've written some tooling using Python/Paramiko to automate SSH-ing into a box and: 1) stop any affected services 2) do a series of puppet updates. I use the new "detailed exit codes" feature in 0.24.6 to determine if we need to run puppet again 3) keep running puppet until the transaction report comes out clean, or we hit an upper bound on the number of attempts 4) if we've reached stability, restart the affected services in reverse order of shutdown The tooling supports use of a semaphore that controls how many machines at a time you'd like to update. For "rolling upgrade" scenarios, we set this number to "1" to update each machine in single file. I've also implemented a "full cycle" operation that will first go to our yum servers, do a puppet update (which makes them pull down the latest rpms for our application and run createrepo, among other things), and if that succeeds we continue updating the rest of the cluster in single-file. For upgrades where we're allowed to take a maintenance window, we just upgrade all the machines simultaneously (since there won't be any active users). So: don't worry; you're not alone. :) deepak --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
