[issue45353] sys.modules: dictionary changed size during iteration

[Terry J. Reedy]

Change by Terry J. Reedy :


--
stage: commit review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45353] sys.modules: dictionary changed size during iteration

[miss-islington]

miss-islington  added the comment:


New changeset 459a4db5eae1f5ef063b34c61cc099820aa9ed0a by Miss Islington (bot) 
in branch '3.10':
bpo-45353: Remind sys.modules users to copy when iterating. (GH-28842)
https://github.com/python/cpython/commit/459a4db5eae1f5ef063b34c61cc099820aa9ed0a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45353] sys.modules: dictionary changed size during iteration

[Gregory P. Smith]

Gregory P. Smith  added the comment:

While arguably unnecessary as it is documented as a dictionary and this is a 
normal Python dict behavior, it is a global dict and it can be modified at 
times that are unintuitive to users of all experience levels.  A note in the 
documentation makes sense.

--
assignee:  -> docs@python
components: +Documentation -Library (Lib)
nosy: +docs@python
resolution:  -> fixed
stage: patch review -> commit review
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45353] sys.modules: dictionary changed size during iteration

[Gregory P. Smith]

Gregory P. Smith  added the comment:


New changeset 3d1ca867ed0e3ae343166806f8ddd9739e568ab4 by Gregory P. Smith in 
branch 'main':
bpo-45353: Remind sys.modules users to copy when iterating. (GH-28842)
https://github.com/python/cpython/commit/3d1ca867ed0e3ae343166806f8ddd9739e568ab4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45353] sys.modules: dictionary changed size during iteration

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +27155
pull_request: https://github.com/python/cpython/pull/28843

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45353] sys.modules: dictionary changed size during iteration

[Gregory P. Smith]

Change by Gregory P. Smith :


--
keywords: +patch
pull_requests: +27154
stage: test needed -> patch review
pull_request: https://github.com/python/cpython/pull/28842

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45353] sys.modules: dictionary changed size during iteration

[Terry J. Reedy]

Terry J. Reedy  added the comment:

In #13487, Gregory fixed the problem by using .copy().  That seems to have 
worked for 1 1/2 years.  You still have not reported an actual bug in the 
current CPython stdlib.

Perhaps we should mention in 
https://docs.python.org/3/library/sys.html#sys.modules that sys.modules can be 
unexpectedly changed during iteration by lazy imports or other threads, so that 
a copy might be needed.

Gregory, what do you think?

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45353] sys.modules: dictionary changed size during iteration

[Idan Cohen]

Idan Cohen  added the comment:

An example can be found here: 
https://github.com/python/cpython/commit/7058d2d96c5ca4dfc6c754c5cd737c6eb2a8fd67
https://bugs.python.org/issue13487

Those links are about an issue that was until March 2020 with how sys.modules 
is iterated. It is not safe to iterate over since it can get updated during the 
iteration because of lazy importing for example.

If you look in older reports you can find that there us an issue with iterating 
over it and it was not addressed yet at all.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45353] sys.modules: dictionary changed size during iteration

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Do you have an example for which "for m in sys.modules: print(m, end=' ')" 
fails?  In particular, is there code in the stdlib that fails iterating over 
sys.modules?  If not the latter, this should be closed as 'Not a bug'.

Note that questions about using Python should go elsewhere, such as python-list 
or stackoverflow.

--
nosy: +terry.reedy
stage:  -> test needed
type:  -> behavior
versions: +Python 3.11 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45353] sys.modules: dictionary changed size during iteration

[Dennis Sweeney]

Dennis Sweeney  added the comment:

One standard way of preventing this is copying the dictionary whenever there's 
risk of it changing out from under you, as in:

modules = sys.modules.copy()
for key, value in modules.items():
...

--
nosy: +Dennis Sweeney

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45353] sys.modules: dictionary changed size during iteration

[Idan Cohen]

New submission from Idan Cohen :

Hi,

When iterating over sys.modules it might be that because of lazy loading or 
that other module reload is being reloaded (even if sys.modules.copy() is being 
used) you will get:

"RuntimeError: dictionary changed size during iteration"

The usage of sys.modules and iteration over it is used in many places

I know that this is a known issue but didn't find any solution or fix for that. 
Also, if there is some work around that might solve this, I would appreciate it.

Thanks,
Idan

--
components: Library (Lib)
messages: 403089
nosy: idan57
priority: normal
severity: normal
status: open
title: sys.modules: dictionary changed size during iteration
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com