[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[Łukasz Langa]

Łukasz Langa  added the comment:

Thanks, Hai Shi! ✨  ✨

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 52d9d3b75441ae6038fadead89eac5eecdd34501 by Łukasz Langa in 
branch '3.9':
[3.9] bpo-44050: Extension modules can share state when they don't support 
sub-interpreters. (GH-27794) (GH-28741)
https://github.com/python/cpython/commit/52d9d3b75441ae6038fadead89eac5eecdd34501


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset d0d29655ffc43d426ad68542d8de8304f7f1346a by Miss Islington (bot) 
in branch '3.10':
bpo-44050: Extension modules can share state when they don't support 
sub-interpreters. (GH-27794) (GH-28738)
https://github.com/python/cpython/commit/d0d29655ffc43d426ad68542d8de8304f7f1346a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[Łukasz Langa]

Change by Łukasz Langa :


--
pull_requests: +27088
pull_request: https://github.com/python/cpython/pull/28741

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[STINNER Victor]

STINNER Victor  added the comment:

Thanks for the fix. I don't understand well this code :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[miss-islington]

miss-islington  added the comment:


New changeset b9bb74871b27d9226df2dd3fce9d42bda8b43c2b by Hai Shi in branch 
'main':
bpo-44050: Extension modules can share state when they don't support 
sub-interpreters. (GH-27794)
https://github.com/python/cpython/commit/b9bb74871b27d9226df2dd3fce9d42bda8b43c2b


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[miss-islington]

Change by miss-islington :


--
pull_requests: +27086
pull_request: https://github.com/python/cpython/pull/28738

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[hai shi]

hai shi  added the comment:

> (Note that the "def->m_size == -1" means that the module does not support 
> sub-interpreters, because it has global state.

Make Sense. It's more better and exact than my suggestion :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[hai shi]

hai shi  added the comment:

> Maybe the old behavior was better: if an extension uses the old API, share 
> its state between all interpreters.

+1. I create PR-27794, which use `def->m_slots` to identify the extension 
module created from `PyModule_Create()` or not. cc @petr

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[Petr Viktorin]

Petr Viktorin  added the comment:

> Maybe the old behavior was better: if an extension uses the old API, share 
> its state between all interpreters.

Yes, I think the old behavior was better: if an extension uses the old API, 
share its state between all interpreters.

This is obviously bad, but I don't see how skipping part of initialization (as 
done in 
https://github.com/python/cpython/commit/82c83bd907409c287a5bd0d0f4598f2c0538f34d#diff-28cfc3e2868980a79d93d2ebdc8747dcb9231f3dd7f2caef96e74107d1ea3bf3L721-R719
 ) is better.
(Note that the "def->m_size == -1" means that the module does not support 
sub-interpreters, because it has global state, per 
https://docs.python.org/3/c-api/module.html#c.PyModuleDef.m_size)

--
nosy: +petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[hai shi]

Change by hai shi :


--
keywords: +patch
pull_requests: +26263
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/27794

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[Trygve Aaberge]

Trygve Aaberge  added the comment:

@jack__d: If you're just getting "no data" once and nothing else, the test 
isn't working correctly. You should get "no data" twice when it works correctly 
and the issue is fixed, or "unknown error" and some other info if the issue is 
present. I'm not sure why it doesn't work for you though.

I tried it with Python 3.9.6 now, and I'm still seeing the issue, with the same 
output as in my original post (apart from a newline between "no data" and 
"unknown error", and a different line number and id numbers).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[Jack DeVries]

Jack DeVries  added the comment:

I just took a look at this, and I'm getting an output of "no data" (just one 
time) on 3.9.6. Has this been fixed?

--
nosy: +jack__d

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[Trygve Aaberge]

Trygve Aaberge  added the comment:

> The _PyImport_FixupExtensionObject() change impacts extensions using the 
> legacy API and so should not be used in subinterpreters.

I'm not using that directly in my code, but I guess it's used indirectly? If it 
shouldn't be used, what's the alternative in 3.9?

> Maybe the old behavior was better: if an extension uses the old API, share 
> its state between all interpreters.

Since this worked fine as far as I know before 3.9, and currently breaks 
existing applications, this is probably better yes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44050] [subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9

[STINNER Victor]

STINNER Victor  added the comment:

> The commit that fixes the issue in 3.10 is 
> https://github.com/python/cpython/commit/7f1305ef9ea7234e1a5aacbea17490232e9b7dc2

Right, the _ssl module was ported to multiphase initialization (PEP 489) in 
bpo-42333 (fixed in Python 3.10.0a3).


> This is a regression in 3.9. It didn't happen in 3.8. The commit that 
> introduced the issue is 
> https://github.com/python/cpython/commit/82c83bd907409c287a5bd0d0f4598f2c0538f34d

In Python 3.9, some stdlib modules are still using the legacy API to initialize 
modules, and share states between interpreters. This is bad: no Python object 
(nor state) must be shared between two interpreters.

Well, there is an on-going work on subinterpreters:

* https://pyfound.blogspot.com/2021/05/the-2021-python-language-summit_16.html
* https://vstinner.github.io/isolate-subinterpreters.html

For example, many C extensions are being converted to the multiphase 
initialization API and get a "module state".

The _PyImport_FixupExtensionObject() change impacts extensions using the legacy 
API and so should not be used in subinterpreters.

Maybe the old behavior was better: if an extension uses the old API, share its 
state between all interpreters.

--
nosy: +corona10, erlendaasland, shihai1991 -christian.heimes
title: Exceptions in a subinterpreter are changed by another subinterpreter -> 
[subinterpreters] _PyImport_FixupExtensionObject() regression in Python 3.9
versions: +Python 3.10, Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com