<cafkp-qzgfj07odylm9m2gj8c1upbdrj7rfxwr02xdtz7zdh...@mail.gmail.com>
On Fri, Oct 28, 2011 at 10:55:08AM +0100, Brian Holt wrote:
> > > Interesting. This hypothesis should be testable, for instance by
> > > keeping a reference on 'a', appending it to a list. I'd be
> > > interested in the results, if you mind trying out Brian.
> > I'm not sure I understand. I thought that by appending to a list I
> > am
> > keeping a reference to the object.
Right, but it seems to me that this is exactly what we want to test the
hyothesis. Maybe I am being dense, as I m a bit rushing through my mail,
but it seems to me that if you keep a reference to a, then you compensate
for the difference that was pointed out in the discussion below, i.e.
that the forest gets garbage-collected after each iteration. Thus I think
that it would enable us to test the hypothesis.
> > > import cPickle
> > >
> > > for i in range(0, 20):
> > > with open("forest%d.pkl" % (i), 'r') as f:
> > > start = datetime.now()
> > > a = cPickle.load(f)
> > > print 'loaded ', i, datetime.now() - start
> > In this case, the reference to `a` is lost after each iteration,
> > which means that the forest can be garbage-collected and that memory
> > can be reused.
> > > [snip]
> > In contrast, in this case, forests can no longer be garbage-collected
> > and new memory need to be allocated at each iteration, the private
> > heap need to be extended and so on. In the process, I suspect that
> > objects are moved to one place to another, which may be the reason
> > why it slows down (since the number of objects in memory keeps
> > increasing, it takes longer and longer to move them).
Gaël
------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
_______________________________________________
Scikit-learn-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
