On 06/07/2013 12:08 AM, Joel Nothman wrote: > I proposed something that did this among a more general solution for > warm starts without memoizing a couple of weeks ago, but I think > memoizing is neater and handles most cases. To handle it generally, > you could add a memoize parameter to Pipeline. Then I guess you'd have > to do some subset of: > * memoize the step estimator for each fit, given its parameters and > the parameters of all preceding estimators, and the input to > Pipeline.fit. (A enhanced version could take advantage of an estimator > specifying that changing certain parameters will affect the result of > transform without refitting.) > * possibly memoize the transformed output for each step estimator > given its parameters and the parameters of all preceding estimators, > and the input to Pipeline.fit. Pipeline methods could then precede by > looking for the latest memoized transform output and start new > calculations from there. > Memorization and parallelization don't play along nicely. You would still need to schedule the estimators so as not to duplicate work, I think :-/
------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
