New submission from Miro Hrončok <m...@hroncok.cz>: Recently, when debugging a weird problem (see https://bugzilla.redhat.com/show_bug.cgi?id=2018551 for details if interested, but not important for this issue), I've realized that the _NamespacePath class (from importlib/_bootstrap_external.py) has a cache that (in some cases) uses tuple(sys.path) as the key. I was expecting importlib.invalidate_caches() to invalidate this cache, but it doesn't.
Consider the following directory structure: . ├── PATH1 │ └── namespace │ └── sub1 │ └── __init__.py └── PATH2 └── namespace └── sub2 └── __init__.py Here is a helper to create it (on Linux-ish): $ mkdir -p PATH1/namespace/sub1 $ mkdir -p PATH2/namespace/sub2 $ touch PATH1/namespace/sub1/__init__.py $ touch PATH2/namespace/sub2/__init__.py Run Python with PYTHONPATH=PATH1:PATH2 (output slightly formatted for readability): $ PYTHONPATH=PATH1:PATH2 python3.11 >>> import namespace >>> namespace.__path__ _NamespacePath(['.../namespace_path_cache/PATH1/namespace', '.../namespace_path_cache/PATH2/namespace']) >>> import namespace.sub1 # works >>> import namespace.sub2 # works >>> exit() The namespace packages seem to work as expected. Now move PATH2/namespace out of the way: $ mv PATH2/namespace PATH2/cant-import-this Run Python again: $ PYTHONPATH=PATH1:PATH2 python3.11 >>> import namespace >>> namespace.__path__ _NamespacePath(['.../namespace_path_cache/PATH1/namespace']) >>> ... While this interpreter still runs, move the PATH2/namespace module back in: $ mv PATH2/cant-import-this PATH2/namespace >>> ... >>> namespace.__path__ _NamespacePath(['.../namespace_path_cache/PATH1/namespace']) >>> import namespace.sub2 Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'namespace.sub2' >>> import importlib >>> importlib.invalidate_caches() # invalidate the cache, not helpful >>> namespace.__path__ _NamespacePath(['.../namespace_path_cache/PATH1/namespace']) >>> import namespace.sub2 Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'namespace.sub2' >>> import sys >>> sys.path.remove('') # changing sys.path solves this >>> namespace.__path__ _NamespacePath(['.../namespace_path_cache/PATH1/namespace']) >>> import namespace.sub2 >>> namespace.__path__ _NamespacePath(['.../namespace_path_cache/PATH1/namespace', '.../namespace_path_cache/PATH2/namespace']) importlib.invalidate_caches() documentation says: > This function should be called if any modules are created/installed while > your program is running to guarantee all finders will notice the new module’s > existence. That makes me think calling importlib.invalidate_caches() should also invalidate the cache of _NamespacePaths. (This also affects older Pythons, but since it is a behavior change, I've only marked 3.11). ---------- messages: 405606 nosy: hroncok, petr.viktorin priority: normal severity: normal status: open title: importlib.invalidate_caches() does not invalidate _NamespacePath's _last_parent_path-based cache type: behavior versions: Python 3.11 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue45703> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com