2016-02-08 15:18 GMT+01:00 Victor Stinner <victor.stin...@gmail.com>: >> Perhaps if you add some guards somewhere :-) > > We have runtime checks but only implemented in debug mode for efficiency. > > By the way, I proposed once to add an environment variable to allow to > enable these checks without having to recompile Python. Since the PEP > 445, it became easy to implement this. What do you think? > https://www.python.org/dev/peps/pep-0445/#add-a-new-pydebugmalloc-environment-variable
Ok, I wrote a patch to implement a new PYTHONMALLOC environment variable: http://bugs.python.org/issue26516 PYTHONMALLOC=debug installs debug hooks to: * detect API violations, ex: PyObject_Free() called on a buffer allocated by PyMem_Malloc() * detect write before the start of the buffer (buffer underflow) * detect write after the end of the buffer (buffer overflow) https://docs.python.org/dev/c-api/memory.html#c.PyMem_SetupDebugHooks The main advantage of this variable is that you don't have to recompile Python in debug mode to benefit of these checks. Recompiling Python in debug mode requires to recompile *all* extensions modules since the debug ABI is incompatible. When I played with tracemalloc on Python 2 ( http://pytracemalloc.readthedocs.org/ ), I had such issues, it was very annoying with non-trivial extension modules like PyQt or PyGTK. With PYTHONMALLOC, you don't have to recompile extension modules anymore! With tracemalloc and PYTHONMALLOC=debug, we will have a complete tool suite to "debug memory"! My motivation for PYTHONMALLOC=debug is to detect API violations to prepare my change on PyMem_Malloc() allocator ( http://bugs.python.org/issue26249 ), but also to help users to detect bugs. It's common that users report a bug: "Python crashed", but have no idea of the responsible of the crash. I hope that detection of buffer underflow & overflow will help them to detect bugs in their own extension modules. Moreover, I added PYTHONMALLOC=malloc to ease the use of external memory debugger on Python. By default, Python uses pymalloc allocator for PyObject_Malloc() which raises a lot of false positive in Valgrind. We even have a configuration (--with-valgrind) and a Valgrind suppressino file to be able to skip these false alarms in Valgrind. IMHO PYTHONMALLOC=malloc is a simpler option to use Valgrind (or other tools). Victor _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
