On 17.07.2018 7:18, Radim Řehůřek wrote:
Hi all,
one of our Python projects calls for pretty heavy, low-level
optimizations.
We went down the rabbit hole and determined that having access to
PyList_GET_ITEM(list), PyInt_AS_LONG(int) and PyDict_GetItem(dict,
unicode) on Python objects **outside of GIL** might be a good-enough
solution. The Python objects in question are guaranteed to live and
not be mutated externally in any way. They're "frozen" and read-only.
The standard practice if you need to access something outside of GIL is
to get a private C object from it.
For immutable types, you can get a pointer to the underlying data if the
internal representation is compatible with some C type (e.g. char*
|PyBytes_AsString|(PyObject/ *o/) and FILE*
|PyFile_AsFile|(PyObject/ *p/) (Py2 only -- in Py3, PyFile no longer
wraps stdio FILE) ); otherwise, the C API can produce a copy (e.g.
wchar_t* |PyUnicode_AsWideCharString|(PyObject/ *unicode/,
Py_ssize_t/ *size/) ).
Though you can call whatever you want outside of GIL (it's not like we
can prevent you), anything that's not officially guaranteed you're doing
on your own risk. Even if it happens to work now, it can break in
unpredictable ways at any point in the future.
Under what conditions is it OK to call these 3 functions on such objects?
More generally, what is the CPython 2.7/3.5 contract regarding (lack
of) object mutation, and the need for reference counting and
synchronization via GIL?
Which C API functions are safe to call on "const" objects?
Obviously releasing GIL and then calling C API is hacky, but from
initial experiments, it seems to work (see
https://stackoverflow.com/questions/51351609/can-i-const-access-cpython-objects-without-gil).
But I'm wondering if there's a more formal contract around this behaviour.
Cheers,
Radim
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru
--
Regards,
Ivan
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
