Re: [Python-Dev] Enhancement of Python memory allocators
2013/6/13 Nick Coghlan : > Yes, that sounds better. One of the biggest problems with the current > startup sequence is the way it relies on environment variables for > configuration, which makes life hard for other applications that want to > embed the CPython runtime. I wrote a new patch (attached to issue #3329) adding a new PyMem_SetupDebugHooks() function. So if an application replaces Python memory allocator functions, it can still can PyMem_SetupDebugHooks() to benefit of the Python debug hooks detecting memory bugs. The function does nothing if hooks are already installed or if Python is not compiled in debug mode. With this function, the API is now complete for all use cases. The PEP 432 helps to configure embedded Python, but the new "Set" functions (ex: PyMem_SetAllocators) are still needed for my pytracemalloc module which installs hooks at runtime, when Python is fully initialized (the hooks can be installed anytime). pytracemalloc is just an example, you may use PyMem_SetAllocators for other debug or performance purpose. With my patch, allocator functions like PyMem_Malloc() are no more macro, and are always the same function. This helps the stable ABI: C extension modules do not need to be recompiled to benefit of the debug hooks ;-) Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ssl improvements and testing question
On Fri, 07 Jun 2013 00:37:01 +0200 Christian Heimes wrote: > > I'm also thinking about OCSP support and X509v3 extension support for > _decode_certificate(). Both are a PITB ... Python has an easier and > better documented C API. Yes, OpenSSL's documentation is often a joke, unfortunately. > Question: > What's the minimum version of OpenSSL Python 3.4 is going to support? Judging by the kind of machines we run on, I would say 0.9.7something. Basically I don't think we should remove any existing #ifdef for 3.4. > Do > we have an easy way to compile and link Python against a custom > installation of OpenSSL or do I have to fiddle around with CPPFLAGS and > CFLAGS? You have to fiddle around, sadly. (and you will also have to fiddle around with LD_LIBRARY_PATH) If you find a way to improve that, your contribution is much welcome :-) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: fixd refleak
On Mon, 10 Jun 2013 10:47:28 +0200 (CEST)
christian.heimes wrote:
> http://hg.python.org/cpython/rev/6860263c05b3
> changeset: 84077:6860263c05b3
> user:Christian Heimes
> date:Mon Jun 10 10:47:22 2013 +0200
> summary:
> fixd refleak
>
[...]
> -return Py_BuildValue("()", ofile_env, ofile, odir_env, odir);
> +if ((tup = PyTuple_New(4)) == NULL) {
> +goto error;
> +}
> +PyTuple_SET_ITEM(tup, 0, ofile_env);
> +PyTuple_SET_ITEM(tup, 1, ofile);
> +PyTuple_SET_ITEM(tup, 2, odir_env);
> +PyTuple_SET_ITEM(tup, 3, odir);
> +return tup;
How about
return Py_BuildValue("", ofile_env, ofile, odir_env, odir);
?
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: fixd refleak
Am 13.06.2013 20:59, schrieb Antoine Pitrou:
> How about
>
> return Py_BuildValue("", ofile_env, ofile, odir_env, odir);
>
> ?
Oh right, I forgot about 'N'. The PyArg_Parse*() methods don't have it.
Do you want me to modify the code to use Py_BuildValue()?
Christian
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: fixd refleak
On Thu, 13 Jun 2013 23:39:49 +0200
Christian Heimes wrote:
> Am 13.06.2013 20:59, schrieb Antoine Pitrou:
> > How about
> >
> > return Py_BuildValue("", ofile_env, ofile, odir_env, odir);
> >
> > ?
>
> Oh right, I forgot about 'N'. The PyArg_Parse*() methods don't have it.
>
> Do you want me to modify the code to use Py_BuildValue()?
Always better to use the simple form, IMO. Go for it!
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Allow calling PyMem_Malloc() without the GIL held in Python 3.4
Hi, I would like to remove the "GIL must be held" restriction from PyMem_Malloc(). In my opinion, the restriction was motived by a bug in Python, bug fixed by the issue #3329. Let me explain why. The PyMem_Malloc() function is a thin wrapper to malloc(). It returns NULL if the size is lager than PY_SSIZE_T_MAX and have a well defined behaviour for PyMem_Malloc(0) (don't return NULL). So it is surprising to read in Include/pymem.h: "The GIL must be held when using these APIs." The reason is more surprising: in debug mode, PyMem_Malloc() is no more a thin wrapper to malloc(), but it calls internally PyObject_Malloc(), the "Python allocator" (called pymalloc). (Many other checks are done in debug mode, but it's unrelated to my point.) The problem is that PyObject_Malloc() is not thread-safe, the GIL must be held. Short history: fb45791150d1 (Mar 23 2002) "gives Python a debug-mode pymalloc" f294fdd18b5b (Mar 28 2002) removes the "check API family" e16dbf875303 (Apr 22 2002) redirects indirectly PyMem_Malloc() to PyObject_Malloc() in debug mode b6aff7a59803 (Sep 28 2009) reintroduces API checks So the GIL issue is almost as old as the debug mode for Python memory allocators. My patch attached to http://bugs.python.org/issue3329 changes the design of the debug memory allocators: they are now wrapper (hooks) on the underlying memory allocator (PyMem: malloc, PyObject: pymalloc), instead of always redirecting to pymalloc (ex: PyObject_Malloc). Using my patch, PyMem_Malloc() now always calls malloc(), even in debug mode. Removing the "GIL must be held" restriction is now safe. Do you agree? May it cause backward compatibility issue? PyMem_Malloc() and PyMem_MALLOC() call malloc(), except if the Python source code was manually modified. Does this use case concern many developers? Removing the GIL restriction would help to replace direct calls to malloc() with PyMem_Malloc(). Using PyMem_SetAllocators(), an application would be able to replace memory allocators, and these allocators would be used "everywhere". => see http://bugs.python.org/issue18203 Victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
