One of the open questions relative to subinterpreters is: how to reduce the amount of work required for extension modules to support them? Thanks to Petr Viktorin for a lot of work he's done in this area (e.g. PEP 489)! Extensions also have the option to opt out of subinterpreter support.
However, that's only one part of the story. A while back Nathaniel expressed concerns with how making subinterpreters more accessible will have a negative side effect affecting projects that publish large extensions, e.g. numpy. Not all extensions support subinterpreters due to global state (incl. in library dependencies). The amount of work to get there may be large. As subinterpreters increase in usage in the community, so will demand increase for subinterpreter support in those extensions. Consequently, such projects be pressured to do the extra work (which is made even more stressful by the short-handed nature of most open source projects) . So we (the core devs) would effectively be requiring those extensions to support subinterpreters, regardless of letting them opt out. This situation has been weighing heavily on my mind since Nathaniel brought this up. Here are some ideas I've had or heard of about what we could do to help: * add a page to the C-API documentation about how to support subinterpreters * identify the extensions most likely to be impacted and offer to help * add more helpers to the C-API to make adding subinterpreter support less painful * fall back to loading the extension in its own namespace (e.g. use ldm_open()) * fall back to copying the extension's file and loading from the copied file * ... I'd appreciate your thoughts on what we can do to help. Thanks! -eric _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/X3ZOSP2A4RTSKTBZ4XYHROSJBONCEDID/ Code of Conduct: http://python.org/psf/codeofconduct/
