This email thread died out but the issue has not been forgotten.  Andy and
I discussed this issue and came up with an initial plan for batch
processing.  Our thinking is to start conservatively and build the feature
out from there.  The new provider interface would be the one Andy suggested:


>   * Provider::batchable?(**resource1, resource2)
>   * Provider::batch_start
>   * Provider::batch_end
>
>
We defined an initial set of vertical slices to work on, with the yum
provider as the initial guinea pig:

1. Define report schema changes
2. Every resource is its own batch. The provider executes the batch and
these batches appear in the report.
3. The provider is able to control what resources can go into a batch to
allow batches of size > 1. Manifest ordering algorithm.
4. The yum provider executes all of the items in a batch using a single
command. It assumes that everything will succeed and no error reporting is
done.
5. The yum provider handles errors while executing a batch and reports a
failure for any item as a failure for all items.
6. The yum provider is able to report which item of the batch caused the
actual error and preserve this information in the report.
7. Extend/test for Title Hash order and Random Order. Prior to this, the
conservative (manifest ordering) algorithm is used per batch.

I suspect we'll learn some about batch processing (and perhaps yum!) as we
go, so these slices aren't set in stone, but just an initial development
plan.  I'm hoping we'll see some of these slices getting developed soon!

Also, I'm dropping this summary in
http://projects.puppetlabs.com/issues/2198 for the benefit of those not
following this thread closely.

Kylo

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-dev@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-dev.
For more options, visit https://groups.google.com/groups/opt_out.

Reply via email to