On Wed, 9 Mar 2016 at 06:57 Victor Stinner <victor.stin...@gmail.com> wrote:
> 2016-02-08 15:18 GMT+01:00 Victor Stinner <victor.stin...@gmail.com>: > >> Perhaps if you add some guards somewhere :-) > > > > We have runtime checks but only implemented in debug mode for efficiency. > > > > By the way, I proposed once to add an environment variable to allow to > > enable these checks without having to recompile Python. Since the PEP > > 445, it became easy to implement this. What do you think? > > > https://www.python.org/dev/peps/pep-0445/#add-a-new-pydebugmalloc-environment-variable > > Ok, I wrote a patch to implement a new PYTHONMALLOC environment variable: > > http://bugs.python.org/issue26516 > > PYTHONMALLOC=debug installs debug hooks to: > > * detect API violations, ex: PyObject_Free() called on a buffer > allocated by PyMem_Malloc() > * detect write before the start of the buffer (buffer underflow) > * detect write after the end of the buffer (buffer overflow) > > https://docs.python.org/dev/c-api/memory.html#c.PyMem_SetupDebugHooks > > The main advantage of this variable is that you don't have to > recompile Python in debug mode to benefit of these checks. > I just wanted to say this all sounds awesome! Thanks for all the hard work on making our memory management story easier to work with, Victor. -Brett > > Recompiling Python in debug mode requires to recompile *all* > extensions modules since the debug ABI is incompatible. When I played > with tracemalloc on Python 2 ( http://pytracemalloc.readthedocs.org/ > ), I had such issues, it was very annoying with non-trivial extension > modules like PyQt or PyGTK. With PYTHONMALLOC, you don't have to > recompile extension modules anymore! > > > With tracemalloc and PYTHONMALLOC=debug, we will have a complete tool > suite to "debug memory"! > > My motivation for PYTHONMALLOC=debug is to detect API violations to > prepare my change on PyMem_Malloc() allocator ( > http://bugs.python.org/issue26249 ), but also to help users to detect > bugs. > > It's common that users report a bug: "Python crashed", but have no > idea of the responsible of the crash. I hope that detection of buffer > underflow & overflow will help them to detect bugs in their own > extension modules. > > > Moreover, I added PYTHONMALLOC=malloc to ease the use of external > memory debugger on Python. By default, Python uses pymalloc allocator > for PyObject_Malloc() which raises a lot of false positive in > Valgrind. We even have a configuration (--with-valgrind) and a > Valgrind suppressino file to be able to skip these false alarms in > Valgrind. IMHO PYTHONMALLOC=malloc is a simpler option to use Valgrind > (or other tools). > > Victor > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
