Dag Sverre Seljebotn, 25.10.2011 15:28:
On 10/25/2011 09:33 AM, Stefan Behnel wrote:
mark florisson, 24.10.2011 21:50:
All of this functionality should also get a sane C API (to be provided
by cython.h). You'd get a Cy_INCREF(obj, have_gil)/Cy_DECREF() etc.
Every class using this functionality is a subclass of CythonObject
(that contains a PyObject + an acquisition count + a lock). Perhaps if
the user is subclassing something other than object we could allow the
user to specify custom __cython_(un)lock__ and
__cython_acquisition_count__ methods and fields.
Now, building on top of this functionality, Cython could provide
built-in nogil-compatible types, like lists, dicts and maybe tuples
(as a start). These will by default not lock for operations to allow
e.g. one thread to iterate over the list and another thread to index
it without lock contention and other general overhead. If one thread
is somehow changing the size of the list, or writing to indices that
another thread is reading from/writing to, the results will of course
be undefined unless the user synchronizes on the object. So it would
be the user's responsibility. The acquisition counting itself will
always be thread-safe (i.e., it will be atomic if possible, otherwise
it will lock).
It's probably best to not enable this functionality by default as it
would be more expensive to instantiate objects, but it could be
supported through a cdef class decorator and a general directive.
It's well known that this would be expensive. One of the approaches that
tried to get rid of the GIL in CPython introduced fine grained locking,
and it turned out to be substantially slower, AFAIR by a factor of two.
I'd gladly take a factor two (or even four) slowdown of CPython code any
day to get rid of the GIL :-). The thing is, sometimes one has 48 cores and
consider a 10x speedup better than nothing...
Ah, sorry, that factor was for single-threaded code. How it would scale for
multi-core code depends on too many factors to make any general statement.
Stefan
_______________________________________________
cython-devel mailing list
cython-devel@python.org
http://mail.python.org/mailman/listinfo/cython-devel
