On Fri, Jan 27, 2012 at 05:10:11PM +0100, Olivier Grisel wrote: > The problem is that for multiprocessing, a n_jobs argument is enough > (to tell the number of cores). But for cluster computing you will have > to pass some kind of active cluster session (e.g. a > IPython.parallel.Client instance that is aware of the configuration of > the engines running on the cluster).
Indeed, that's a problem. > Maybe it's just a matter or renaming "n_jobs" by something more generic > such as "workers" and make joblib smart enough to introspect this guy > with some ducktyping to address the parallel case and fallback on it's > on multiprocessing backend if the workers object is not an IPython > Client or View instance. That's one way. I was also thinking that, in addition to such an enriched semantic, a dispatcher mechanism in joblib, in which you would be able to register pools (that could very well be described by an IPython.parallel.Client instance) might be handy, to avoid having to pass along such arguments in the code. G ------------------------------------------------------------------------------ Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
