[issue31517] MainThread association logic is fragile

[STINNER Victor]

Change by STINNER Victor :


--
components: +Subinterpreters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Eric Snow]

Eric Snow  added the comment:

probably a duplicate: issue #39042 "Use the runtime's main thread ID in the 
threading module."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[STINNER Victor]

STINNER Victor  added the comment:

bpo-37416 has been marked as a duplicate of this issue. It contains snippet.py 
to reproduce a bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[STINNER Victor]

STINNER Victor  added the comment:

Python internals already know who is the "main" thread: _PyRuntime.main_thread. 
It's maintained up to date, even after a fork, PyOS_AfterFork_Child() calls 
_PyRuntimeState_ReInitThreads() which does:

// This was initially set in _PyRuntimeState_Init().
runtime->main_thread = PyThread_get_thread_ident();

I already added _thread._is_main_interpreter() to deny spawning daemon threads 
in subinterpreters: bpo-37266.

We can add _thread._is_main_thread() which can reuse Modules/signalmodule.c 
code:

static int
is_main(_PyRuntimeState *runtime)
{
unsigned long thread = PyThread_get_thread_ident();
PyInterpreterState *interp = 
_PyRuntimeState_GetThreadState(runtime)->interp;
return (thread == runtime->main_thread
&& interp == runtime->interpreters.main);
}

For example, this function is used by signal.signal:

if (!is_main(runtime)) {
PyErr_SetString(PyExc_ValueError,
"signal only works in main thread");
return NULL;
}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +aldwinaldwin, fabioz, int19h, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Antoine Pitrou]

Change by Antoine Pitrou :


--
versions: +Python 3.8, Python 3.9 -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Nick Coghlan]

Nick Coghlan added the comment:

We had the quirks of import related threading deadlocks documented for a long 
time, not as a promise, but as a debugging aid (and a recommendation for "don't 
do that").

I'd see this as being similar: we'd document that if you're using the _thread 
module, or otherwise allowing operating system threads not managed by the 
threading module to call in and run arbitrary Python code, then you should run 
"import threading" early in the actual main thread to make sure it gets 
associated correctly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Antoine Pitrou]

Antoine Pitrou added the comment:

I'm not sure this is an issue worth fixing, but I don't think it's a good idea 
to document such quirky semantics.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Antoine Pitrou]

Antoine Pitrou added the comment:

> That said, I do agree with Tim that the status quo isn't broken per se: we 
> renamed `thread` to `_thread` in Python 3 for a reason

This is not caused by `_thread` specifically, but any background thread created 
by C code that might invoke Python code.  The reproducer just uses the 
`_thread` module out of convenience.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Antoine Pitrou]

Antoine Pitrou added the comment:

> Is there a problem here?  I haven't heard of anyone even wondering about this 
> before.

This came while working on the PEP 556 implementation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Nick Coghlan]

Nick Coghlan added the comment:

One place where this came up recently is in working out precisely how a 
Python-level subinterpreter API will interact with the threading API: 
https://mail.python.org/pipermail/python-dev/2017-September/149566.html

That said, I do agree with Tim that the status quo isn't broken per se: we 
renamed `thread` to `_thread` in Python 3 for a reason, and that reason is that 
you really need to know how Python's threading internals work to do it safely.

However, I do think we can treat this as a documentation enhancement request, 
where the `_thread` module docs could point out some of the requirements to 
ensure that low-level thread manipulation plays nice with the threading module.

--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python
type: behavior -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Tim Peters]

Tim Peters added the comment:

Is there a problem here?  I haven't heard of anyone even wondering about this 
before.  threading.py worries about Threads created _by_ threading.py, plus the 
magical (from its point of view) thread that first imports threading.py.  Users 
mix in `_thread` threads, or raw C threads from extension modules, essentially 
at their own risk.  Which risks are minimal, but can have visible consequences.

I don't view that as being a real problem.  It might help if, e.g., a Wiki 
somewhere spelled out the consequences under different Python implementations 
(while I don't know for sure, I _expect_ the current docs just say "in normal 
conditions, the main thread is the thread from which the Python interpreter was 
started" because it doesn't want to over-specify the behavior in an area nobody 
really cares about).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Antoine Pitrou]

Antoine Pitrou added the comment:

Fixing this will require thinking out what the "main thread" of an interpreter 
really is.  Is it the thread in which PyInterpreterState_New() is called?

(note this is not the same thing as the "main thread" in the signal module, 
which is process-global)

--
nosy: +eric.snow, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +asvetlov, tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31517] MainThread association logic is fragile

[Antoine Pitrou]

New submission from Antoine Pitrou:

The threading main_thread() instance is associated to the thread that first 
imports the threading module.  In usual circumstances, this will indeed be the 
interpreter's main thread.  However, it is not always the case.  See attached 
reproducer.

$ ./python mainthread2.py 
child thread: <_MainThread(MainThread, started 140399567398656)>
main thread: <_DummyThread(Dummy-1, started daemon 140399588386560)>
Exception ignored in: 
Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/threading.py", line 1268, in _shutdown
assert tlock.locked()
AssertionError:

--
files: mainthread2.py
messages: 302521
nosy: pitrou
priority: normal
severity: normal
stage: needs patch
status: open
title: MainThread association logic is fragile
type: behavior
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file47153/mainthread2.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com