I commited the new API (little bit different than my last patch on issue #3329): http://hg.python.org/cpython/rev/6661a8154eb3
The documentation will be available in a few minutes at: http://docs.python.org/3/c-api/memory.html 2013/6/14 Kristján Valur Jónsson <krist...@ccpgames.com>: >> Removing the GIL restriction would help to replace direct calls to >> malloc() with PyMem_Malloc(). Using PyMem_SetAllocators(), an application >> would be able to replace memory allocators, and these allocators would be >> used "everywhere". >> => see http://bugs.python.org/issue18203 > > To keep this interesting, I have a somewhat different opinion to Victor :) > have put comments in the original defect, but would like to repeat them > here. > IMHO, keeping the GIL restriction on PyMem_MALLOC is useful. > 1) It allows it to be replaced with PyObject_MALLOC(). Or anything else. In > particular, an implementer is free to add memory profiling support and other > things without worrying about implementation details. Requiring it to be GIL > free severely limits what it can do. For example, it would be forbidden to > delegate to PyObject_MALLOC when debugging. For my own pytracemalloc tool, holding the GIL while calling PyMem_Malloc() is required to be able to retrieve the Python filename and line number of the caller. So you convinced me :-) I am also worried by the backward compatibility, even if I expect that only a very few developers replaced Python memory allocators. A custom memory allocator may not be thread-safe, so the GIL can also be convinient. I added new functions in the "final" API: PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree(). These functions are just wrapper for malloc(), realloc() and free(). The GIL does not need to be hold. No process is done before/after at all. Behaviour of PyMem_RawMalloc(0) is platform depend for example. "size > PY_SSIZE_T_MAX" check is not done neither, but it may be interesting to add this check for security reasons (it is already in place for PyMem_Malloc and PyObject_Malloc). Using these new functions instead of malloc/realloc/free is interesting because the internal functions can be replaced with PyMem_SetRawAllocators() and many checks are added in debug mode (ex: check for buffer under- and overflow). PyObject_Malloc() was not documented, so I did not document PyObject_SetAllocators(). In the final API, I added a new PyMemAllocators structure to simplify the API. I also made _PyObject_SetArenaAllocators() private because I don't like its API (it is not homogenous with PyMem_SetAllocators) and it is concerned by less use cases. I prefer to wait a little before making this API public. I didn't use "#ifndef Py_LIMITED_API", so all new functions are part of the stable API. Is it correct? Victor _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
