On Mar 17, 2009, at 2:07 PM, Mark Plaksin wrote: > > On Mar 16, 12:51pm, Mark Plaksin <[email protected]> wrote: >> On Mar 10, 2:49am, James Turnbull <[email protected]> wrote: >> >>> Puppet 0.24.8 Release Candidate 1 is now available for testing. >>> >>> Enhancements to file serving, module and stored configuration >>> performance >> >> Even with these changes we can't run storeconfigs. It makes mysqld >> eat >> all of our CPUs and cry for more. We even stopped puppetmaster, >> dropped >> all of the tables to remove any cruft, and restarted puppetmaster. > > Thanks to lots of help from Brice we now have all of our hosts in our > storeconfigs database. The big hurdle was the initial INSERTs of data > from our 400 clients. Besides MySQL tuning we ended up turning > puppetd off on half of our hosts. Once data for the other half made > it into the database, we ran 'puppetd --test --tags smallmodule' on > the other half to get them in the database. It looks like that's > going to do the trick for us. > > It would be nice if there were an easier way to get your foot in the > door :) Also, if another project comes along that has 200 hosts we'll > have to be keep storeconfigs in mind as we bring them up.
I'll hopefully be sending the plan to the list today, but for a customer we're working on a queueing system to help mitigate this. All of the catalog saves would be queued and processed as time was available, so if you had a lot of big changes, they'd happen as cpu time was available, rather than killing all of your clients like they do now. We also need to start adding back noop optimizations that test whether the catalog actually needs to be stored or even compiled; we removed them all last year because we couldn't consistently make decisions about whether inputs had changed, thus necessitating a recompile. -- The easiest way for your children to learn about money is for you not to have any. -- Katharine Whitehorn --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
