Re: [Scikit-learn-general] Problems with Python's garbage collection in GridSearch

2014-12-16 Thread Sebastian Raschka
I really have no idea, but thanks for the suggestion. Unfortunately, gc.isenabled() evaluates to True. > On Dec 16, 2014, at 2:05 PM, Lars Buitinck wrote: > > 2014-12-16 17:57 GMT+01:00 Sebastian Raschka : >> Maybe it's something in scipy according to Manoj's linked discussion ... in >> any c

Re: [Scikit-learn-general] Problems with Python's garbage collection in GridSearch

2014-12-16 Thread Lars Buitinck
2014-12-16 17:57 GMT+01:00 Sebastian Raschka : > Maybe it's something in scipy according to Manoj's linked discussion ... in > any case, maybe a workaround for this issue and future issues would be to > have a > "forxe_clear_gc" (default=False) parameter to force the garbage collector to > be em

Re: [Scikit-learn-general] Problems with Python's garbage collection in GridSearch

2014-12-16 Thread Gael Varoquaux
On Tue, Dec 16, 2014 at 11:57:47AM -0500, Sebastian Raschka wrote: > Maybe it's something in scipy according to Manoj's linked discussion ... in > any case, maybe a workaround for this issue and future issues would be to > have a > "forxe_clear_gc" (default=False) parameter to force the garbage

Re: [Scikit-learn-general] Problems with Python's garbage collection in GridSearch

2014-12-16 Thread Sebastian Raschka
Hi, Andy, the models that I am using are Random Forests and naive Bayes classifiers. Maybe it's something in scipy according to Manoj's linked discussion ... in any case, maybe a workaround for this issue and future issues would be to have a "forxe_clear_gc" (default=False) parameter to force t

Re: [Scikit-learn-general] Problems with Python's garbage collection in GridSearch

2014-12-16 Thread Andy
Hi. Which models are you using and which version of scikit-learn? Cheers, Andy On 12/16/2014 11:19 AM, Sebastian Raschka wrote: > Hi all, > > I am wondering if someone noticed that GridSearch is eating more and more > memory over time? I read related discussion on the issue list on GitHub and

Re: [Scikit-learn-general] Problems with Python's garbage collection in GridSearch

2014-12-16 Thread Manoj Kumar
Hi, Is there are a slight possibility that this is related to this issue? ( https://github.com/scikit-learn/scikit-learn/issues/3762) . This was an issue with Scikit-learn solvers that are wrappers around SciPy's solvers, (eg, sparse-cg). They are unable to garbage-collect memory due to reference

[Scikit-learn-general] Problems with Python's garbage collection in GridSearch

2014-12-16 Thread Sebastian Raschka
Hi all, I am wondering if someone noticed that GridSearch is eating more and more memory over time? I read related discussion on the issue list on GitHub and it sounds like that it has been solved (estimators are not kept anymore, and the best estimator can optionally be refitted at the end of