New submission from Daniel Colascione <dan...@google.com>:

Right now, the main loop in semlock_acquire looks like this:

    do {
        Py_BEGIN_ALLOW_THREADS
        if (blocking && timeout_obj == Py_None)
            res = sem_wait(self->handle);
        else if (!blocking)
            res = sem_trywait(self->handle);
        else
            res = sem_timedwait(self->handle, &deadline);
        Py_END_ALLOW_THREADS
        err = errno;
        if (res == MP_EXCEPTION_HAS_BEEN_SET)
            break;
    } while (res < 0 && errno == EINTR && !PyErr_CheckSignals());

Here, we unconditionally release the GIL even we could acquire the mutex 
without blocking! As a result, we could end up switching to another thread in 
the process and greatly increasing the latency of operations that lock and 
release multiple shared data structures.

Instead, we should unconditionally try sem_trywait, and only then, if we want 
to block, release the GIL and try sem_wait or sem_timedwait. This way, we'll 
churn only when we need to.

Note that threading.Lock works the way I propose.

----------
components: Library (Lib)
messages: 303441
nosy: Daniel Colascione
priority: normal
severity: normal
status: open
title: Don't release the GIL if we can acquire a multiprocessing semaphore 
immediately
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue31653>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Reply via email to