That sounds awesome, Kylo. If, in the course of implementing this, you happen to find it easy to implement an easy way to delay resources that might not be "ready", that would be awesome, too. That is, if there were a hook similar to those below that could be used to see if a resource is ready to run, then resources that aren't ready could be put into the back of the queue. This would be useful for things like waiting to bring up services until a remote required service is ready.
I expect this is too large a change to shoehorn into this effort, but I figured I'd at least bring it up. The biggest difference (both this and batching involve delaying resources) is that this puts you in a place where you might only have non-ready resources remaining, which is a new kind of failure state, whereas batching packages should never result in that. Either way, I look forward to this batching showing up. On Oct 14, 2013, at 4:18 PM, Kylo Ginsberg <k...@puppetlabs.com> wrote: > This email thread died out but the issue has not been forgotten. Andy and I > discussed this issue and came up with an initial plan for batch processing. > Our thinking is to start conservatively and build the feature out from there. > The new provider interface would be the one Andy suggested: > > > * Provider::batchable?(resource1, resource2) > * Provider::batch_start > * Provider::batch_end > > > We defined an initial set of vertical slices to work on, with the yum > provider as the initial guinea pig: > > 1. Define report schema changes > 2. Every resource is its own batch. The provider executes the batch and these > batches appear in the report. > 3. The provider is able to control what resources can go into a batch to > allow batches of size > 1. Manifest ordering algorithm. > 4. The yum provider executes all of the items in a batch using a single > command. It assumes that everything will succeed and no error reporting is > done. > 5. The yum provider handles errors while executing a batch and reports a > failure for any item as a failure for all items. > 6. The yum provider is able to report which item of the batch caused the > actual error and preserve this information in the report. > 7. Extend/test for Title Hash order and Random Order. Prior to this, the > conservative (manifest ordering) algorithm is used per batch. > > I suspect we'll learn some about batch processing (and perhaps yum!) as we > go, so these slices aren't set in stone, but just an initial development > plan. I'm hoping we'll see some of these slices getting developed soon! > > Also, I'm dropping this summary in http://projects.puppetlabs.com/issues/2198 > for the benefit of those not following this thread closely. > > Kylo > > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-dev+unsubscr...@googlegroups.com. > To post to this group, send email to puppet-dev@googlegroups.com. > Visit this group at http://groups.google.com/group/puppet-dev. > For more options, visit https://groups.google.com/groups/opt_out. -- http://puppetlabs.com/ | http://about.me/lak | @puppetmasterd -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To post to this group, send email to puppet-dev@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-dev. For more options, visit https://groups.google.com/groups/opt_out.
