[issue45183] Unexpected exception with zip importer

[Brett Cannon]

Brett Cannon  added the comment:

I decided that find_spec() saying something wasn't available in a finder made 
sense even though it was because a zip file no longer existed as the loader 
would still fail as appropriate.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Brett Cannon]

Brett Cannon  added the comment:


New changeset e1bdecb6dc7ac33256d5fa875d45634512d2a90e by Brett Cannon in 
branch '3.10':
[3.10] bpo-45183: don't raise an exception when calling 
zipimport.zipimporter.find_spec() when the zip file is missing and the internal 
cache has been reset (GH-28435) (#28438)
https://github.com/python/cpython/commit/e1bdecb6dc7ac33256d5fa875d45634512d2a90e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Brett Cannon]

Change by Brett Cannon :


--
pull_requests: +26843
pull_request: https://github.com/python/cpython/pull/28438

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Brett Cannon]

Brett Cannon  added the comment:


New changeset 209b7035f714dcc41df054b0b023e0b955d7e1a2 by Brett Cannon in 
branch 'main':
bpo-45183: don't raise an exception when calling 
zipimport.zipimporter.find_spec() when the zip file is missing and the internal 
cache has been reset (GH-28435)
https://github.com/python/cpython/commit/209b7035f714dcc41df054b0b023e0b955d7e1a2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[miss-islington]

Change by miss-islington :


--
keywords: +patch
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +26842
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28437

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Brett Cannon]

Brett Cannon  added the comment:

The proposed fix seems to be the right one based on my reading of the code.

--
keywords:  -patch
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Brett Cannon]

Change by Brett Cannon :


--
keywords: +patch
pull_requests: +26840
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28435

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Ronald Oussoren]

Ronald Oussoren  added the comment:

I just noticed that I'm unnecessarily obtuse in my description of a possible 
fix, the diff (without test update):

% git diff Lib/zipimport.py 
   (main)cpython
diff --git a/Lib/zipimport.py b/Lib/zipimport.py
index c55fec6aa1..43ac6cbe57 100644
--- a/Lib/zipimport.py
+++ b/Lib/zipimport.py
@@ -334,7 +334,7 @@ def invalidate_caches(self):
 _zip_directory_cache[self.archive] = self._files
 except ZipImportError:
 _zip_directory_cache.pop(self.archive, None)
-self._files = None
+self._files = {}
 
 
 def __repr__(self):


With that change the exception should not happen, and the now stale zipimporter 
would be ignored when flushing the cache while the zipfile referenced by the 
zip importer instance has been removed.

That said, I haven't tested this and won't create a PR because my local tree is 
(still) a mess.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Brett Cannon]

Change by Brett Cannon :


--
assignee:  -> brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Brett, can you take a look when you have some time?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I bisected this to:

3abf6f010243a91bf57cbf357dac33193f7b8407 is the first bad commit
commit 3abf6f010243a91bf57cbf357dac33193f7b8407
Author: Desmond Cheong 
Date:   Tue Mar 9 04:06:02 2021 +0800

bpo-14678: Update zipimport to support importlib.invalidate_caches() 
(GH-24159)



Added an invalidate_caches() method to the zipimport.zipimporter class 
based on the implementation of importlib.FileFinder.invalidate_caches(). This 
was done by adding a get_files() method and an _archive_mtime attribute to 
zipimport.zipimporter to check for updates or cache invalidation whenever the 
cache of files and toc entry information in the zipimporter is accessed.

 Doc/library/zipimport.rst  |9 +
 Lib/test/test_zipimport.py |   41 +
 Lib/zipimport.py   |   10 +
 .../2021-01-07-21-25-49.bpo-14678.1zniCH.rst   |3 +
 Python/importlib_zipimport.h   | 1896 ++--
 5 files changed, 1020 insertions(+), 939 deletions(-)
 create mode 100644 
Misc/NEWS.d/next/Library/2021-01-07-21-25-49.bpo-14678.1zniCH.rst
bisect run success


Which is https://bugs.python.org/issue14678

--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Ronald Oussoren]

Change by Ronald Oussoren :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Ronald Oussoren]

Change by Ronald Oussoren :


--
nosy: +pablogsal, twouters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45183] Unexpected exception with zip importer

[Ronald Oussoren]

New submission from Ronald Oussoren :

The attached file demonstrates the problem:

If importlib.invalidate_caches() is called while the zipfile used by the zip 
importer is not available the import system breaks entirely. I found this in a 
testsuite that accedently did this (it should have updated sys.path). 

I get the following exception:

$ python3.10 t.py
Traceback (most recent call last):
  File "/Users/ronald/Projects/modulegraph2/t.py", line 27, in 
import uu
  File "", line 1027, in _find_and_load
  File "", line 1002, in _find_and_load_unlocked
  File "", line 945, in _find_spec
  File "", line 1430, in find_spec
  File "", line 1402, in _get_spec
  File "", line 168, in find_spec
  File "", line 375, in _get_module_info
TypeError: argument of type 'NoneType' is not iterable


This exception is not very friendly

This particular exception is caused by setting self._files to None in the 
importer's invalidate_caches method, while not checking for None in 
_get_modules_info. 

I'm not sure what the best fix would be, setting self._files to an empty list 
would likely be the easiest fix.

Note that the script runs without errors in Python 3.9.

--
files: repro.py
keywords: 3.10regression
messages: 401698
nosy: ronaldoussoren
priority: normal
severity: normal
status: open
title: Unexpected exception with zip importer
versions: Python 3.10, Python 3.11
Added file: https://bugs.python.org/file50279/repro.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com