On Wed, Aug 24, 2022 at 4:42 AM Petr Viktorin <encu...@gmail.com> wrote: > On 23. 08. 22 16:19, Sebastian Berg wrote: > > Our public C-API is currently exported as a single static struct into > > the library loading NumPy. If types depend on the interpreter, it > > would seem we need to redo the whole mechanism? > > Right, sounds like it needs to be a dynamically allocated struct. > In the interim, one instance of the struct is static: that's the one > used for anything that doesn't support multiple interpreters yet, and > also as the module state in one “main” module object. (That would be the > first module to be loaded, and until everything switches over, it'd get > an unpaired incref to become “immortal” and leak at exit.) > > > Further, many of the functions would need to be adapted. We might be > > able to hack that the API looks the same [1]. However, it cannot be > > ABI compatible, so we would need a whole new API table/export mechnism > > and some sort of shim to allow compiling against older NumPy versions > > but using it with all versions (otherwise we need 2+ years of > > patience). > > Having one static “main” module state in the interim would also help here. > > > Of course there might be a point in saying that most C-API use is > > initially not subinterpreter ready, but it does seem like a pretty huge > > limitation... > > A huge limitation, but it might be a good way to break up the work to > make it more manageable :)
FWIW, in CPython there's a similar issue. We currently expose static pointers to all the builtin exceptions in the C-API. Even worse, we expose the object *values* for all the static types and the several singletons. On top of that, these are all exposed in the limited API (stable ABI). As a result, moving to one each per interpreter is messy. PEP 684 talks about the possible solutions. The simplest for us is to make all those objects immortal. However, in some cases we also have to do an interpreter-specific lookup internally. I expect you would have to do similar where/when compatibility remains essential. -eric _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
