Re: memory recycling/garbage collecting problem

2009-02-26 Thread David Niergarth
On Feb 16, 11:21 pm, Yuanxin Xi xi11w...@gmail.com wrote: Could anyone please explain why this happens?  It seems some memory are not freed. There is a bug in versions of Python prior to 2.5 where memory really isn't released back to the OS. Python 2.5 contains a new object allocator that is

Re: memory recycling/garbage collecting problem

2009-02-26 Thread David Niergarth
Tim Peters showed a way to demonstrate the fix in http://mail.python.org/pipermail/python-dev/2006-March/061991.html For simpler fun, run this silly little program, and look at memory consumption at the prompts: x = [] for i in xrange(100): x.append([]) raw_input(full ) del x[:]

Re: memory recycling/garbage collecting problem

2009-02-26 Thread Steve Holden
David Niergarth wrote: Tim Peters showed a way to demonstrate the fix in http://mail.python.org/pipermail/python-dev/2006-March/061991.html For simpler fun, run this silly little program, and look at memory consumption at the prompts: x = [] for i in xrange(100): x.append([])

Re: memory recycling/garbage collecting problem

2009-02-26 Thread Terry Reedy
Steve Holden wrote: David Niergarth wrote: Tim Peters showed a way to demonstrate the fix in http://mail.python.org/pipermail/python-dev/2006-March/061991.html For simpler fun, run this silly little program, and look at memory consumption at the prompts: x = [] for i in xrange(100):

Re: memory recycling/garbage collecting problem

2009-02-26 Thread Steve Holden
Terry Reedy wrote: Steve Holden wrote: David Niergarth wrote: [...] I'm not sure what deleting a slice accomplishes (del x[:]); the behavior is the same whether I do del x or del x[:]. Any ideas? del x removes the name x from the current namespace, garbage collecting the object to which it

Re: memory recycling/garbage collecting problem

2009-02-18 Thread Tim Wintle
On Tue, 2009-02-17 at 17:04 +0100, Christian Heimes wrote: Tim Wintle wrote: Basically malloc() and free() are computationally expensive, so Python tries to call them as little as possible - but it's quite clever at knowing what to do - e.g. if a list has already grown large then python

Re: memory recycling/garbage collecting problem

2009-02-17 Thread Aaron Brady
On Feb 16, 11:21 pm, Yuanxin Xi xi11w...@gmail.com wrote: I'm having some problems with the memory recycling/garbage collecting of the following testing code: a=[str(i) for i in xrange(1000)] This takes 635m/552m/2044 memory (VIRT/RES/SHR) b={} for i in xrange(1000): ...    

Re: memory recycling/garbage collecting problem

2009-02-17 Thread Chris Rebert
On Tue, Feb 17, 2009 at 12:33 AM, Aaron Brady castiro...@gmail.com wrote: On Feb 16, 11:21 pm, Yuanxin Xi xi11w...@gmail.com wrote: I'm having some problems with the memory recycling/garbage collecting of the following testing code: a=[str(i) for i in xrange(1000)] This takes

Re: memory recycling/garbage collecting problem

2009-02-17 Thread Tim Wintle
On Tue, 2009-02-17 at 00:40 -0800, Chris Rebert wrote: 'gc.collect()' -- I believe, but I'm not the specialist in it. If I understand correctly, that only effects objects that are part of a reference cycle and doesn't necessarily force the freed memory to be released to the OS. I

Re: memory recycling/garbage collecting problem

2009-02-17 Thread Floris Bruynooghe
On Feb 17, 5:31 am, Chris Rebert c...@rebertia.com wrote: My understanding is that for efficiency purposes Python hangs on to the extra memory even after the object has been GC-ed and doesn't give it back to the OS right away. Even if Python would free() the space no more used by it's own

Re: memory recycling/garbage collecting problem

2009-02-17 Thread Christian Heimes
Yuanxin Xi wrote: Could anyone please explain why this happens? It seems some memory are not freed. I'm running into problems with this as my program is very memory cosuming and need to frequently free some object to reuse the memory. What is the best way to free the memory of b completely

Re: memory recycling/garbage collecting problem

2009-02-17 Thread Christian Heimes
Tim Wintle wrote: Basically malloc() and free() are computationally expensive, so Python tries to call them as little as possible - but it's quite clever at knowing what to do - e.g. if a list has already grown large then python assumes it might grow large again and keeps hold of a percentage

memory recycling/garbage collecting problem

2009-02-16 Thread Yuanxin Xi
I'm having some problems with the memory recycling/garbage collecting of the following testing code: a=[str(i) for i in xrange(1000)] This takes 635m/552m/2044 memory (VIRT/RES/SHR) b={} for i in xrange(1000): ... b[str(i)]=i Then the memory usage increased to 1726m/1.6g/2048

Re: memory recycling/garbage collecting problem

2009-02-16 Thread Chris Rebert
On Mon, Feb 16, 2009 at 9:21 PM, Yuanxin Xi xi11w...@gmail.com wrote: I'm having some problems with the memory recycling/garbage collecting of the following testing code: a=[str(i) for i in xrange(1000)] This takes 635m/552m/2044 memory (VIRT/RES/SHR) b={} for i in xrange(1000):

Re: memory recycling/garbage collecting problem

2009-02-16 Thread Marc 'BlackJack' Rintsch
On Mon, 16 Feb 2009 23:21:16 -0600, Yuanxin Xi wrote: I'm having some problems with the memory recycling/garbage collecting of the following testing code: a=[str(i) for i in xrange(1000)] This takes 635m/552m/2044 memory (VIRT/RES/SHR) b={} for i in xrange(1000): ...