Re: [Scikit-learn-general] GridSearchCV using a ton of memory

2014-10-10 Thread Chris Holdgraf
I recreated it with some fake data, here's a short gist to show what I'm talking about: http://nbviewer.ipython.org/gist/choldgraf/6a7be7866f2a3a3d3f98 On Fri, Oct 10, 2014 at 1:29 PM, Chris Holdgraf wrote: > Yes - this is with the latest scikit-learn. Also, I'm using n_jobs==1, so > there shou

Re: [Scikit-learn-general] Best way to integrate probability density function from GMM? -- Fixed

2014-10-10 Thread Robert Kern
On 2014-10-10 00:10, Benjamin Blumer wrote: > Hi all, > > I see that gmm.score(x) returns the log probability of x for that > point. I'm interested in integrating this probability over a region. > For example, finding the probability of a ball being in the space > (x,y,z) +/- (delta_x, delta_y, de

Re: [Scikit-learn-general] GridSearchCV using a ton of memory

2014-10-10 Thread Chris Holdgraf
Yes - this is with the latest scikit-learn. Also, I'm using n_jobs==1, so there shouldn't be any memmapping anyway, right? Which version of scikit-learn? Have you tried with 0.15.2? Your data should automatically get memory mapped to share some input data with the 'n_jobs' worker processes. --

Re: [Scikit-learn-general] GridSearchCV using a ton of memory

2014-10-10 Thread Olivier Grisel
Which version of scikit-learn? Have you tried with 0.15.2? Your data should automatically get memory mapped to share some input data with the 'n_jobs' worker processes. -- Olivier -- Meet PCI DSS 3.0 Compliance Requirem

[Scikit-learn-general] GridSearchCV using a ton of memory

2014-10-10 Thread Chris Holdgraf
Hey all - I'm running into some memory issues with GridSearchCV and I wonder if anyone can give an intuition as to why. I'm cross-validating alpha parameters for Ridge regression. I'm trying 8 different parameters. My inputs are 2400x1900 (~370MB) in size. When I run %memit model.fit(X, y) alo