En Mon, 26 Oct 2009 23:42:54 -0300, s7v7nislands s7v7nisla...@gmail.com
escribió:
On Oct 27, 4:03 am, Diez B. Roggisch de...@nospam.web.de wrote:
s7v7nislands schrieb:
test.py
#!/usr/bin/python
a = []
for i in xrange(100):
a.append('a'*500)
$python -i test.py #virt mem 514m in top output
del a #virt mem 510m
why python cache these string?
In source, I see when object size SMALL_REQUEST_THRESHOLD, python
would use malloc.
also use free() when string refcount == 0.
do I miss somethong?
http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-d...
thanks. but it can't explain why cache string size bigger than
SMALL_REQUEST_THRESHOLD.
In source, It seems only allocate from cache pool when object's size
SMALL_REQUEST_THRESHOLD (256),
when size SMALL_REQUEST_THRESHOLD, it will direct use malloc.
see void *PyObject_Malloc(size_t nbytes) in Objects/obmalloc.c
I know the range xrange's memory use. the int,str object has their
memory manager above Python's object allocator.
but why python don't free big string's memory, which size
SMALL_REQUEST_THRESHOLD
It does - it isn't Python who keeps those memory blocks, but the C stdlib.
Apparently free doesn't bring the memory block back to the OS, but keeps
itself internally as free blocks.
If you repeat the cycle again, you should see that memory usage doesn't
grow beyond the previous value; when those big strings are malloc()ed
again, the C stdlib fulfills the memory request using those free blocks it
originally got from the OS.
This is what I see on Windows (using the pslist utility from sysinternals):
# memory usage right when Python starts:
D:\USERDATA\Gabrielpslist -m python
PsList 1.23 - Process Information Lister
Copyright (C) 1999-2002 Mark Russinovich
Sysinternals - www.sysinternals.com
Process memory detail for LEPTON:
Name Pid VM WS WS PkPriv Faults NonP Page
PageFile
python 3672 32916473247322828 12122 55
2828
# after creating the big list of strings
python 3672 622892 533820 533820 531724 1417372 55
531724
# after `del a`
python 3672 6187126656 5338284212 1417472 55
4212
# after re-creating the list
python 3672 622960 533844 533844 531732 2810372 55
531732
# after `del a` again, repeating a few times
python 3672 6187126708 5338444244 2810372 55
4244
python 3672 6187126672 5338484220 4203172 55
4220
python 3672 6187126696 5338484252 5596082 55
4252
python 3672 6187126680 5338484228 6988912 55
4228
Note the 'VM' and 'Priv' columns (virtual address space and private bytes,
respectively).
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
