Hi Allessandro,
you might want to look into this presentation by Olivier
https://speakerdeck.com/ogrisel/growing-randomized-trees-in-the-cloud-1 --
it should be pretty much what you need. Code is here
https://github.com/pydata/pyrallel.
best,
Peter
2014-02-07 23:28 GMT+01:00 Alessandro Gagliardi <
[email protected]>:
> Hi All,
>
> I want to run a large sklearn.ensemble.RandomForestClassifier (with
> maybe a dozens or maybe hundreds of trees and 100,000 samples). My desktop
> won't handle this so I want to try using StarCluster.
> RandomForestClassifier seems to parallelize easily, but I don't know how I
> would split it across many IPython.parallel engines (if that's even
> possible). (Or maybe I should be foregoing IPython.parallel and using MPI?)
>
> Any help would be greatly appreciated.
>
> Thanks,
>
> Alessandro Gagliardi| Glassdoor| [email protected]
>
> *We're hiring! Check out our open jobs
> <http://www.glassdoor.com/about/careers.htm>.*
>
> *Twitter <https://twitter.com/Glassdoor>** | Facebook
> <https://www.facebook.com/Glassdoor> | Glassdoor Blog
> <http://www.glassdoor.com/blog/>*
>
> *2012 Webby Award Winner: Best Employment Site*
>
> *2013 Webby Award Winner: Best Guides/Ratings/Review Site*
>
>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
> _______________________________________________
> Scikit-learn-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
--
Peter Prettenhofer
------------------------------------------------------------------------------
Managing the Performance of Cloud-Based Applications
Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
Read the Whitepaper.
http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
