Hi Joris, On Mon, 2019-08-19 at 17:35 +0200, Joris Van den Bossche wrote: > Hi all, > > PyGEOS (https://github.com/caspervdw/pygeos) is an experimental > package implementing a set of numpy ufuncs to provide vectorized > geometry functionality (wrapping the C++ GEOS library). > > The way it does this is by implementing a Python extension type > (pygeos.Geometry) that wraps an actual GEOSGeometry object by storing > a pointer to it in the PyObject struct of the extension type. This > way, we can store those objects in an object dtype array in numpy, > but still access the pointer in the ufunc inner loop without needing > the python interpreter. > > The single threaded performance of the ufuncs with the approach above > is very good. There doesn't seem to be an overhead of using the > object array approach. > However, as far as I can find in the docs, the GIL is only released > in ufuncs for non-object dtypes. > > So the question here is: is there a way to let numpy release the GIL > in such a case nonetheless? Although the array holds python objects, > the ufunc inner loop only accesses a static attribute of the object > struct, not needing any explicit Python interaction. >
Hmmm, interesting use case. No, I do not think there currently is a reasonable way to do this (I think there may be ways to hack it). Even when all access to the objects is safe by itself, you still have the problem that the object stored inside the array could be replaced (and invalidated) at any time if you run multithreaded. We would like to type such objects in the future, even then, I am not sure how to make things safe against race conditions if elements are replaced (and deleted). This is an interesting use case, since arrays of pointers (or specific pyobjects) will always have this type of issue, and I am not sure how you would avoid it (a cheap lock on the object itself works probably, but even if it is cheap, it is probably fairly expensive?). Best, Sebastian > Best, > Joris > _______________________________________________ > NumPy-Discussion mailing list > [email protected] > https://mail.python.org/mailman/listinfo/numpy-discussion
signature.asc
Description: This is a digitally signed message part
_______________________________________________ NumPy-Discussion mailing list [email protected] https://mail.python.org/mailman/listinfo/numpy-discussion
