[issue45021] Race condition in thread.py

[nullptr]

nullptr  added the comment:

Simplifying the reproducing example a bit more:


from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
from time import sleep

def submit(pool):
pool.submit(submit, pool)


if __name__ == '__main__':
pool = ThreadPoolExecutor(1)
pool.submit(submit, pool)

while True:
with ProcessPoolExecutor() as workers:
print('WORK')
workers.submit(sleep, 0.01).result()
print('DONE')
print('OK')

--

___
Python tracker 
<https://bugs.python.org/issue45021>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45021] Race condition in thread.py

[nullptr]

nullptr  added the comment:

I don't think so: this issue would only arise when using fork on linux and not 
spawn (win/osx) like it seems your system is doing

--

___
Python tracker 
<https://bugs.python.org/issue45021>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45021] Race condition in thread.py

[nullptr]

nullptr  added the comment:

A more direct way to reproduce


from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
from time import sleep

def worker():
with ProcessPoolExecutor() as pool:
r = list(pool.map(sleep, [0.01] * 8))


def submit(pool):
pool.submit(submit, pool)


if __name__ == '__main__':
pool = ThreadPoolExecutor(2)
pool.submit(submit, pool)

i = 0
while True:
r = pool.submit(worker)
r = r.result()
print(i)
i += 1

--

___
Python tracker 
<https://bugs.python.org/issue45021>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45021] Race condition in thread.py

[nullptr]

Change by nullptr :


--
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue45021>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45021] Race condition in thread.py

[nullptr]

New submission from nullptr :

The following code can sometimes hang up


import random 
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
from time import sleep

def worker():   
with ProcessPoolExecutor() as pool:  
r = list(pool.map(sleep, [0.01] * 8))   

   
if __name__ == '__main__': 
pool = ThreadPoolExecutor() 
   
i = 0  
while True:
if random.random() < 0.9:
pool.submit(sleep, 0.001)  
else: 
r = pool.submit(worker) 
r = r.result()
i += 1   
print('alive', i)


It's a bit hard to trigger that way but with some luck and many restarts it'll 
eventually freeze as r.result() never returns.

The backtrace from a child process shows that the child is stuck in 
Lib/concurrent/futures/thread.py:_python_exit waiting for _global_shutdown_lock.

The fork happened while the lock was already grabbed i.e. while executing 
ThreadPoolExecutor.submit

--
components: Library (Lib)
messages: 400378
nosy: xavier.lacroze
priority: normal
severity: normal
status: open
title: Race condition in thread.py
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue45021>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com