[issue40442] Spurious warning emitted during fork() can deadlock a multi-threaded process

[Bowie Chen]

Change by Bowie Chen :


--
nosy: +bowiechen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40442] Spurious warning emitted during fork() can deadlock a multi-threaded process

[Ionel Cristian Mărieș]

Change by Ionel Cristian Mărieș :


--
nosy: +ionelmc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40442] Spurious warning emitted during fork() can deadlock a multi-threaded process

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40442] Spurious warning emitted during fork() can deadlock a multi-threaded process

[Deomid Ryabkov]

Change by Deomid Ryabkov :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40442] Spurious warning emitted during fork() can deadlock a multi-threaded process

[Deomid Ryabkov]

New submission from Deomid Ryabkov :

I know, I know - forking a multi-threaded process is bad. But it happens.

This is related  to https://bugs.python.org/issue40399 and 
https://bugs.python.org/issue6721 but is distinct because the problem is 
entirely self-inflicted.

What happens:

1) A multithreaded program forks using one of the functions, such as 
os.forkpty()
2) In the child process the Python interpreter, in its PyOS_AfterFork_Child 
function ([1]) tries to kill all the threads other than the one doing the 
forking.
3) Among the objects being destroyed may include file or socket objects that 
are now being destroyed too, without having been previosuly closed, which 
triggers a ResourceWarning in the finalizer [2], [3].
4) Default action for warnings is to write to sys.stderr
5) A mutex used in BufferedIO is held by some other (now deceased thread).
6) Deadlock in _enter_buffered_busy [4].

This is bad because there is absolutely no way to avoid it without disabling 
warnings.
Even if the program is super careful to not do anything after forking other 
than exec, it doesn't help because the resource warning and the resulting 
deadlock is triggered by activity of the interpreter: it is the interpreter 
that orphans and is forcibly destroying the files and sockets, not the program 
that lost track of them.

[1] 
https://github.com/python/cpython/blob/beba1a808000d5fc445cb28eab96bdb4cdb7c959/Modules/posixmodule.c#L451

[2] 
https://github.com/python/cpython/blob/beba1a808000d5fc445cb28eab96bdb4cdb7c959/Modules/_io/fileio.c#L95

[3] 
https://github.com/python/cpython/blob/beba1a808000d5fc445cb28eab96bdb4cdb7c959/Modules/socketmodule.c#L4800

[4] 
https://github.com/python/cpython/blob/beba1a808000d5fc445cb28eab96bdb4cdb7c959/Modules/_io/bufferedio.c#L282

--
components: Library (Lib)
messages: 367701
nosy: rojer
priority: normal
severity: normal
status: open
title: Spurious warning emitted during fork() can deadlock a multi-threaded 
process
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com