[issue26003] Issues with PyEval_InitThreads and PyGILState_Ensure

[STINNER Victor]

STINNER Victor  added the comment:

The GIL is now created by _PyEval_InitGIL() which is called by Py_Initialize(): 
at Python startup.

_PyRuntime.main_thread is set by _PyRuntime_Initialize() which is also called 
by Py_Initialize(): at Python startup.

The code to handle signals and pending calls changed a lot since 2016. I close 
the issue.

--
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26003] Issues with PyEval_InitThreads and PyGILState_Ensure

[Irit Katriel]

Irit Katriel  added the comment:

Since PyEval_InitThreads is now an empty function and to be removed, this issue 
seems out of date. A I right?

--
nosy: +iritkatriel
resolution:  -> out of date
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26003] Issues with PyEval_InitThreads and PyGILState_Ensure

[SilentGhost]

Changes by SilentGhost :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26003] Issues with PyEval_InitThreads and PyGILState_Ensure

[tzickel]

Changes by tzickel :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26003] Issues with PyEval_InitThreads and PyGILState_Ensure

[tzickel]

New submission from tzickel:

A few issues regarding threads:
A. (Python 2 & 3) The documentation (https://docs.python.org/3/c-api/init.html) 
about initializing the GIL/Threading system does not specify that calling 
PyEval_InitThreads actually binds the calling thread as the main_thread in the 
ceval.c, meaning that the thread will be in charge till the process goes down 
for handling Py_AddPendingCall calls, and if it ends/dies, they won't be 
handled anymore.

This ceval.c's main_thread is different BTW from the one in signalmodule.c 
which is bound to the thread that called Py_InitializeEx.

Maybe there is sense for both main_thread to be the same one and initialized in 
the same time ? (even without a GIL existing)

B. (Python 3) Besides the bad documentation regarding this, python 3.4 added 
issue #19576 which actually hides the call for PyEval_InitThreads inside 
PyGILState_Ensure. Without careful care and knowledge by the programmer, this 
might cause for a short lived thread created in C to actually bind the 
ceval.c's main_thread and when the thread dies main_thread will never be 
changed again.

The reason this is important is beforehand, the programmer needed to think 
about PyEval_InitThreads now it's hidden and not even mentioned in the 
documentation.

C. (Python 2 & 3) In PyEval_InitThreads documentation it's written "It is not 
safe to call this function when it is unknown which thread (if any) currently 
has the global interpreter lock." Thus it should be mentioned that 
PyGILState_Ensure is now calling it in the documentation ?

Also I believe the reason this quote exists is because a potential race 
condition between thread A which might be running code in PyEval_EvalFrameEx 
(before PyEval_InitThreads is called, and thus is not GIL aware), and thread B 
which calls PyEval_InitThreads then calls PyGILState_Ensure, then running 
Python code, while thread A is still running python code as well. I think it 
should be explained more clearly in the documentation the implications (race 
condition).

I think there might be a way to make an PyEval_InitThreads variant which can 
overcome this race condition. Basically it involves using Py_AddPendingCall to 
a C function which calls PyEval_InitThreads, and notifies the calling 
command/thread when it's done. This way we can be sure that the GIL is taken by 
one thread, and all the others are blocked. (maybe a signal should be sent as 
well, in case the main_thread is blocked on an I/O operation).

D. (Python 2) If the main_thread finishes it's job, while other python threads 
are still alive, signal handling isn't processed anymore (Example will be added 
as a file).

--
components: Interpreter Core
files: signalexample.py
messages: 257425
nosy: tzickel
priority: normal
severity: normal
status: open
title: Issues with PyEval_InitThreads and PyGILState_Ensure
versions: Python 2.7, Python 3.6
Added file: http://bugs.python.org/file41485/signalexample.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com