[issue31653] Don't release the GIL if we can acquire a multiprocessing semaphore immediately

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 828ca59208af0b1b52a328676c5cc0c5e2e999b0 by Victor Stinner in 
branch 'master':
bpo-31653: Remove deadcode in semlock_acquire() (#4091)
https://github.com/python/cpython/commit/828ca59208af0b1b52a328676c5cc0c5e2e999b0


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31653] Don't release the GIL if we can acquire a multiprocessing semaphore immediately

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +4061

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31653] Don't release the GIL if we can acquire a multiprocessing semaphore immediately

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset c872d39d324cd6f1a71b73e10406bbaed192d35f by Antoine Pitrou in 
branch 'master':
bpo-31653: Don't release the GIL if we can acquire a multiprocessing semaphore 
immediately (#4078)
https://github.com/python/cpython/commit/c872d39d324cd6f1a71b73e10406bbaed192d35f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31653] Don't release the GIL if we can acquire a multiprocessing semaphore immediately

[Antoine Pitrou]

Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31653] Don't release the GIL if we can acquire a multiprocessing semaphore immediately

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +4048
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31653] Don't release the GIL if we can acquire a multiprocessing semaphore immediately

[Antoine Pitrou]

Change by Antoine Pitrou :


--
type: enhancement -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31653] Don't release the GIL if we can acquire a multiprocessing semaphore immediately

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Looks reasonable to me. Do you want to provide a patch Daniel?

--
components: +Extension Modules -Library (Lib)
keywords: +easy (C)
nosy: +davin, pitrou, serhiy.storchaka
stage:  -> needs patch
type:  -> enhancement
versions:  -Python 3.4, Python 3.5, Python 3.6, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31653] Don't release the GIL if we can acquire a multiprocessing semaphore immediately

[Daniel Colascione]

New submission from Daniel Colascione :

Right now, the main loop in semlock_acquire looks like this:

do {
Py_BEGIN_ALLOW_THREADS
if (blocking && timeout_obj == Py_None)
res = sem_wait(self->handle);
else if (!blocking)
res = sem_trywait(self->handle);
else
res = sem_timedwait(self->handle, );
Py_END_ALLOW_THREADS
err = errno;
if (res == MP_EXCEPTION_HAS_BEEN_SET)
break;
} while (res < 0 && errno == EINTR && !PyErr_CheckSignals());

Here, we unconditionally release the GIL even we could acquire the mutex 
without blocking! As a result, we could end up switching to another thread in 
the process and greatly increasing the latency of operations that lock and 
release multiple shared data structures.

Instead, we should unconditionally try sem_trywait, and only then, if we want 
to block, release the GIL and try sem_wait or sem_timedwait. This way, we'll 
churn only when we need to.

Note that threading.Lock works the way I propose.

--
components: Library (Lib)
messages: 303441
nosy: Daniel Colascione
priority: normal
severity: normal
status: open
title: Don't release the GIL if we can acquire a multiprocessing semaphore 
immediately
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com