[issue47195] importlib lock race issue in deadlock handling code

[Richard Purdie]

Richard Purdie  added the comment:

This is a production backtrace after I inserted code to traceback if tid was 
already in _blocking_on. It is being triggered by a warning about an unclosed 
asyncio event loop and confirms my theory about nested imports, in the 
production case I'd guess being triggered by gc given the __del__.

  File 
"/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/classes/base.bbclass",
 line 26, in oe_import
import oe.data
  File "", line 1024, in _find_and_load
  File "", line 171, in __enter__
  File 
"/home/pokybuild/yocto-worker/oe-selftest-fedora/build/bitbake/lib/bb/cooker.py",
 line 168, in acquire
return orig_acquire(self)
  File "", line 110, in acquire
  File "/usr/lib64/python3.10/asyncio/base_events.py", line 685, in __del__
_warn(f"unclosed event loop {self!r}", ResourceWarning, source=self)
  File "/usr/lib64/python3.10/warnings.py", line 112, in _showwarnmsg
_showwarnmsg_impl(msg)
  File "/usr/lib64/python3.10/warnings.py", line 28, in _showwarnmsg_impl
text = _formatwarnmsg(msg)
  File "/usr/lib64/python3.10/warnings.py", line 128, in _formatwarnmsg
return _formatwarnmsg_impl(msg)
  File "/usr/lib64/python3.10/warnings.py", line 56, in _formatwarnmsg_impl
import tracemalloc
  File "", line 1024, in _find_and_load
  File "", line 171, in __enter__
  File 
"/home/pokybuild/yocto-worker/oe-selftest-fedora/build/bitbake/lib/bb/cooker.py",
 line 167, in acquire
bb.warn("\n".join(traceback.format_stack()))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue47195] importlib lock race issue in deadlock handling code

[Richard Purdie]

New submission from Richard Purdie :

We've seen tracebacks in production like:

  File "", line 1004, in 
_find_and_load(name='oe.gpg_sign', import_=)
  File "", line 158, in 
_ModuleLockManager.__enter__()
  File "", line 110, in _ModuleLock.acquire()
 KeyError: 139622474778432

and

  File "", line 1004, in 
_find_and_load(name='oe.path', import_=)
  File "", line 158, in 
_ModuleLockManager.__enter__()
  File "", line 110, in _ModuleLock.acquire()
 KeyError: 140438942700992

I've attached a reproduction script which shows that if an import XXX is in 
progress and waiting at the wrong point when an interrupt arrives (in this case 
a signal) and triggers it's own import YYY, _blocking_on[tid] in 
importlib/_bootstrap.py gets overwritten and lost, triggering the traceback we 
see above upon exit from the second import.

I'm using a signal handler here as the interrupt, I don't know what our 
production source is as yet but this reproducer proves it is possible.

--
components: Interpreter Core
files: testit2.py
messages: 416517
nosy: rpurdie
priority: normal
severity: normal
status: open
title: importlib lock race issue in deadlock handling code
versions: Python 3.10
Added file: https://bugs.python.org/file50714/testit2.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com