>> What do you think would be the hard part / why >> do you think this is much work? >> > I think that their is a bit of learning to be done. In particular, the > code would have to be well tested, and I have no idea of what the right > way to test it would be. The second problem would be to devise the > extensive test suite, and be convinced that they are no race conditions > when gathering the results. I have found that this is tricky code. > > I belief that testing might be REALLY tricky. I was more going for a prototype solution than for a releasable at the moment.
>> I have no idea about these things and not much of an idea of how joblib >> works but changing the backends looks not-so-hard from my naive point >> of view. >> > No, probably not. The 2 main bodies of work would be: > > 1. To extract out the pool management code from the joblib.Parallel > object to be able to use other kinds of pools than the > multiprocessing pool. > > 2. To implement a dispatch mechanism to be able to decide which pool the > Parallel object gets. > > None of these sound terribly hard, but that's still a bit of effort to > make it work reliably. > > In the past, when developping such changes in joblib with a specific > scikit-learn driven usecase, I have done a branch of scikit-learn > replacing the local copy of joblib by an import to the real joblib (you > can look at how it is doen in the debian branch, for instance > https://github.com/scikit-learn/scikit-learn/tree/debian/sklearn/externals > ) > Then, I branched out joblib, and made my changes in the joblib branch, > testing them on real-world usecases with the scikit. > > Thanks for the hint. I was just doing that ;) Cheers, Andy ------------------------------------------------------------------------------ Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
