Antoine Pitrou added the comment:
``./configure --without-pymalloc'' fails here:
gcc -pthread -Xlinker -export-dynamic -o python Modules/python.o
libpython3.4.a -lpthread -ldl -lutil -lm
libpython3.4.a(sysmodule.o): In function `sys_getallocatedblocks':
STINNER Victor added the comment:
sys.gettotalrefcount() is only defined when Python is compiled in
debug mode. sys.getallocatedblocks() should only be available when the
right debug option is present. This function is specific to CPython
anyway, Python module should not rely on this (too much)
anatoly techtonik added the comment:
Memory control over the stuff that Python creates is a practical feature that
compensates OS disability for tracking memory usage. If all Python scripts
could measure their memory usage, we could see more memory effective and
adaptive programs around.
Antoine Pitrou added the comment:
sys.gettotalrefcount() is only defined when Python is compiled in
debug mode. sys.getallocatedblocks() should only be available when
the
right debug option is present. This function is specific to CPython
anyway, Python module should not rely on this (too
Stefan Krah added the comment:
Antoine Pitrou rep...@bugs.python.org wrote:
Hmm, interesting. When built --without-pymalloc, we could make
sys.getallocatedblocks() always return 0, or we could not define it all
(which would make like a bit harder for regrtest). What do you think?
Given the
Roundup Robot added the comment:
New changeset a85673b55177 by Antoine Pitrou in branch 'default':
Following issue #13390, fix compilation --without-pymalloc, and make
sys.getallocatedblocks() return 0 in that situation.
http://hg.python.org/cpython/rev/a85673b55177
--
Stefan Krah added the comment:
``./configure --without-pymalloc'' fails here:
gcc -pthread -Xlinker -export-dynamic -o python Modules/python.o
libpython3.4.a -lpthread -ldl -lutil -lm
libpython3.4.a(sysmodule.o): In function `sys_getallocatedblocks':
Roundup Robot added the comment:
New changeset c40f4c19d20b by Antoine Pitrou in branch 'default':
Issue #13390: New function :func:`sys.getallocatedblocks()` returns the number
of memory blocks currently allocated.
http://hg.python.org/cpython/rev/c40f4c19d20b
--
nosy: +python-dev
Antoine Pitrou added the comment:
Committed and pushed!
--
resolution: - fixed
stage: patch review - committed/rejected
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13390
Antoine Pitrou added the comment:
Minor bikeshedding/spelling nit: should
_Py_AllocedBlocks
be changed to
_Py_AllocatedBlocks
(and s/_Py_GetAllocedBlocks/_Py_GetAllocatedBlocks/)?
Right indeed. I'll do the change.
--
___
Python tracker
Antoine Pitrou added the comment:
As it turns out, ctypes does leak memory: see issue 16628.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13390
___
Antoine Pitrou added the comment:
Updated patch with doc. If noone objects, I will commit soon.
--
Added file: http://bugs.python.org/file28227/debugblocks6.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13390
Dave Malcolm added the comment:
Minor bikeshedding/spelling nit: should
_Py_AllocedBlocks
be changed to
_Py_AllocatedBlocks
(and s/_Py_GetAllocedBlocks/_Py_GetAllocatedBlocks/)?
--
___
Python tracker rep...@bugs.python.org
Changes by Antoine Pitrou pit...@free.fr:
--
versions: +Python 3.4 -Python 3.3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13390
___
___
Antoine Pitrou added the comment:
Here is an updated patch.
test_ctypes still leaks memory blocks with it:
$ ./python -m test -R 3:8 test_ctypes
[1/1] test_ctypes
beginning 11 repetitions
12345678901
...
test_ctypes leaked [2, 2, 1, 1, 1, 1, 1, 1] memory blocks, sum=10
--
nosy:
anatoly techtonik techto...@gmail.com added the comment:
How different is the performance cost of this solution compared to inserting
DTrace probe for the same purpose?
--
nosy: +techtonik
___
Python tracker rep...@bugs.python.org
STINNER Victor victor.stin...@haypocalc.com added the comment:
How different is the performance cost of this solution compared
to inserting DTrace probe for the same purpose?
DTrace is only available on some platforms (Solaris and maybe FreeBSD?).
--
Changes by Jesús Cea Avión j...@jcea.es:
--
nosy: +jcea
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13390
___
___
Python-bugs-list mailing list
anatoly techtonik techto...@gmail.com added the comment:
On Mon, Dec 12, 2011 at 6:11 PM, STINNER Victor rep...@bugs.python.orgwrote:
STINNER Victor victor.stin...@haypocalc.com added the comment:
How different is the performance cost of this solution compared
to inserting DTrace probe
Meador Inge mead...@gmail.com added the comment:
I looked at the 'ctypes' leak a bit. I haven't determined exactly what
is going on, but the leak has something to do with a change in the patch that
runs 'dash_R_cleanup' twice instead of once. The new behavior can be reduced
to something like:
Antoine Pitrou pit...@free.fr added the comment:
So, I am not sure whether there really is a leak, if this is just
a very senstive area of 'regrtest.py', or something else I am missing.
Well, test_ctypes seems to be the only test exhibiting that behaviour.
And since your script reproduces it,
Changes by Dave Malcolm dmalc...@redhat.com:
--
nosy: +dmalcolm
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13390
___
___
Python-bugs-list
Antoine Pitrou pit...@free.fr added the comment:
The feature is interesting, but I'm not convinced that a very simple
counter is enough to track memory leaks. It may help the CPython test
suite, but what about real world application?
Good question. A simple counter is the only thing we can
Meador Inge mead...@gmail.com added the comment:
On Fri, Dec 2, 2011 at 12:12 PM, Antoine Pitrou rep...@bugs.python.org wrote:
yes, c6dafa2e2594 was found with this patch. It's the only one, though
(there's also a leak in test_ctypes but I don't want to investigate :-)).
I'll take a look at
Charles-François Natali neolo...@free.fr added the comment:
If someone finds a way to sanitize Valgrind output,
Did you use the valgrind suppression file (Misc/valgrind-python.supp)?
If yes, then one could simply use --gen-suppressions=yes to add those spurious
warning to the suppression
Antoine Pitrou pit...@free.fr added the comment:
If someone finds a way to sanitize Valgrind output,
Did you use the valgrind suppression file (Misc/valgrind-python.supp)?
Ah, I hadn't. But using it doesn't seem to make much of a difference.
Also, the suppressions file seems quite
STINNER Victor victor.stin...@haypocalc.com added the comment:
The overhead on PyObject_Malloc() is just an increment on an integer, so it is
very low (or null).
The feature is interesting, but I'm not convinced that a very simple counter is
enough to track memory leaks. It may help the
Charles-François Natali neolo...@free.fr added the comment:
Together with -R, it can help chase those memory leaks which aren't
reference leaks (see c6dafa2e2594).
Valgrind does a much better job at this: it will also show you where the leaked
blocks were allocated.
OTOH, Valgrind is
Nick Coghlan ncogh...@gmail.com added the comment:
This will likely be a decent you have a problem indicator, but you may still
need tools like Valgrind to actually track down the *cause* of that problem.
--
___
Python tracker rep...@bugs.python.org
Antoine Pitrou pit...@free.fr added the comment:
Valgrind does a much better job at this: it will also show you where
the leaked blocks were allocated.
OTOH, Valgrind is Linux-only and slow, but since I haven't used the
'-R' option much, I don't know how usable this will be in practice
Antoine Pitrou pit...@free.fr added the comment:
Thanks for the comments, here is a new patch addressing them.
I've kept the C API available in all builds (since it's private), but
sys.getallocatedblocks() is only available in debug builds.
As for the memory leak run results, I think we may
Changes by Antoine Pitrou pit...@free.fr:
Added file: http://bugs.python.org/file23680/debugblocks2.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13390
___
Changes by Antoine Pitrou pit...@free.fr:
Added file: http://bugs.python.org/file23681/debugblocks3.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13390
___
Antoine Pitrou pit...@free.fr added the comment:
And the latest patch (debugblocks3.patch) adds said heuristic.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13390
___
New submission from Antoine Pitrou pit...@free.fr:
This patch adds a counting of the number of allocated memory blocks (through
the PyObject_Malloc API). Together with -R, it can help chase those memory
leaks which aren't reference leaks (see c6dafa2e2594).
The sys.getallocedblocks() function
Changes by Andreas Stührk andy-pyt...@hammerhartes.de:
--
nosy: +Trundle
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13390
___
___
Nick Coghlan ncogh...@gmail.com added the comment:
I added some review comments to the patch, but I'm not sure how usable this is
going to be in practice. References generally stay fairly stable while using
the interactive interpreter, but the new block accounting jumps around all over
the
37 matches
Mail list logo
