Hi All, I've been undertaking a refactor of the directoryservice provider on OS X for about a month or so, and I have a couple of branches with different scenarios:
1. No prefetching (self.instances for `puppet resource` and separate getter/setter methods for every property) --> https://github.com/glarizza/puppet-1/blob/feature/master/osx_dscl_providers/lib/puppet/provider/user/directoryservice.rb 2. Prefetching (self.instances for `puppet resource` and self.prefetch for resource prefetching) --> https://github.com/glarizza/puppet-1/blob/feature/osx_dscl_providers/optimization/lib/puppet/provider/user/directoryservice.rb What I'm unsure of is whether it's better to use prefetching and front-load all Puppet runs while the provider prefetches every instance of the type on the system, or whether to use separate getter/setter methods for every property and forgo prefetching. Here's the pros and cons of each as I see it: Prefetching pros: * I can make one call to dscl for each user that will pre-populate every current property value, whereas separate getter methods would require several dscl calls for every user * The `puppet resource user` call completes about 2x to 2.5x faster (since it only makes a max of two dscl calls per user) * Puppet catalogs with many users will be enforced faster since property values are queried at the start of the run and with only a max of 2 dscl system calls (versus a dscl system call for every property) * Makes @property_hash available in all methods for the current property values Prefetching cons: * On a blank 10.7 system with about 8 users, a manifest with two user resources that are in-sync takes about 4.5 seconds to complete a catalog run (due to prefetching every user on the system for every Puppet run) * Potential for stale data - If an initial Puppet run (with many packages/files/services) takes about 2 - 5 minutes or more, then there's a chance that the user values prefetched at the front of the run could be stale (packages creating users, manual changes on the system, etc) if a user needs to be declared at the end of a run. Yes, the errors would be caught and handled, but there's still that consideration Separate getter/setter pros: * More readable - if you need to trace what's happening in the provider, it's easier to trace back the code you're looking for. * Puppet catalogs with a fewer number of users are enforced faster in a Puppet run since you don't need to prefetch every instance of the provider and only need to check the properties of the users in the catalog. Separate getter/setter cons: * The `puppet resource user ` command takes about 2x to 2.5x slower (or greater with more users) since it has to make a dscl system call for every property value on every user on the system Some data: Puppet apply on a catalog with 2 users and multiple properties: * prefetching: sudo puppet apply usertest.pp --trace 11.06s user 6.19s system 79% cpu 21.631 total * no prefetching: sudo puppet apply usertest.pp --trace 4.95s user 2.80s system 60% cpu 12.760 total Puppet resource: * prefetching: sudo puppet resource user --trace 8.74s user 4.29s system 105% cpu 12.392 total * no prefetching: sudo puppet resource user --trace 44.37s user 17.32s system 108% cpu 56.629 total I'm sure there are more pros/cons, but those are the gist of the issues I'm wrestling with. The issue here is that I can't query flat-files on the system for property values - I have to use dscl for everything. All the setter methods are going to have dscl calls regardless of whether I'm prefetching or not (in the future I could probably collect every resource change and use self.flush to make a single dscl call for every resource that has pending changes and perform multiple property changes at once). I'm leaning toward prefetching since the getter method calls would become excessive for multiple users in a single catalog, but I thought I'd pose the question to see what others think (or where my logic is flawed). -- Gary Larizza Professional Services Engineer Puppet Labs Join us for PuppetConf 2012 at the Mission Bay Convention Center in San Francisco, California on September 27th and 28th --> http://bit.ly/pcsig12 -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
