Re: [Python-Dev] Python initialization and embedded Python

I proposed a PR to explicitly list functions safe to be called before Py_Initialize(): https://bugs.python.org/issue32124 https://github.com/python/cpython/pull/4540 I found more than 11 functions.. I also found variables ;-) Victor 2017-11-24 5:01 GMT+01:00 Nick Coghlan

Re: [Python-Dev] Python initialization and embedded Python

24.11.17 04:21, Glenn Linderman пише: On 11/23/2017 5:31 PM, Nick Coghlan wrote: - a consolidated list of *all* the APIs that can safely be called before Py_Initialize So it is interesting to know that list, of course, but the ones that are to be supported and documented might be a smaller

Re: [Python-Dev] Python initialization and embedded Python

On 24 November 2017 at 12:21, Glenn Linderman wrote: > On 11/23/2017 5:31 PM, Nick Coghlan wrote: > > - a consolidated list of *all* the APIs that can safely be called before > Py_Initialize > > So it is interesting to know that list, of course, but the ones that are > to

Re: [Python-Dev] Python initialization and embedded Python

On 11/23/2017 5:31 PM, Nick Coghlan wrote: - a consolidated list of *all* the APIs that can safely be called before Py_Initialize So it is interesting to know that list, of course, but the ones that are to be supported and documented might be a smaller list. Or might not.

Re: [Python-Dev] Python initialization and embedded Python

On 24 November 2017 at 09:19, Victor Stinner wrote: > Hi, > > We are close to the 3.7a3 release and the bug is not fixed yet. I > propose to revert the changes on memory allocators right now, and take > time to design a proper fix which will respect all constraints. > >

Re: [Python-Dev] Python initialization and embedded Python

Hi, We are close to the 3.7a3 release and the bug is not fixed yet. I propose to revert the changes on memory allocators right now, and take time to design a proper fix which will respect all constraints. https://github.com/python/cpython/pull/4532 Today, someone came to me on IRC to complain

Re: [Python-Dev] Python initialization and embedded Python

On Thu, 23 Nov 2017 10:37:59 +0100 "M.-A. Lemburg" wrote: > On 18.11.2017 01:01, Victor Stinner wrote: > > Hi, > > > > The CPython internals evolved during Python 3.7 cycle. I would like to > > know if we broke the C API or not. > > > > Nick Coghlan and Eric Snow are working on

Re: [Python-Dev] Python initialization and embedded Python

On 18.11.2017 01:01, Victor Stinner wrote: > Hi, > > The CPython internals evolved during Python 3.7 cycle. I would like to > know if we broke the C API or not. > > Nick Coghlan and Eric Snow are working on cleaning up the Python > initialization with the "on going" PEP 432: >

Re: [Python-Dev] Python initialization and embedded Python

On 22 November 2017 at 21:12, Victor Stinner wrote: > 2017-11-22 12:04 GMT+01:00 Antoine Pitrou : > > IMHO this really needs a simple solution documented somewhere. Also, > > hopefully when you do the wrong thing, you get a clear error message to >

Re: [Python-Dev] Python initialization and embedded Python

On Wed, 22 Nov 2017 12:12:32 +0100 Victor Stinner wrote: > 2017-11-22 12:04 GMT+01:00 Antoine Pitrou : > > IMHO this really needs a simple solution documented somewhere. Also, > > hopefully when you do the wrong thing, you get a clear error message

Re: [Python-Dev] Python initialization and embedded Python

2017-11-22 12:04 GMT+01:00 Antoine Pitrou : > IMHO this really needs a simple solution documented somewhere. Also, > hopefully when you do the wrong thing, you get a clear error message to > know how to fix your code? Right now, calling PyMem_RawMalloc() before calling

Re: [Python-Dev] Python initialization and embedded Python

On Wed, 22 Nov 2017 10:38:32 +0100 Victor Stinner wrote: > > I fixed the issue by calling _PyRuntime_Initialize() as the very first > function in main(). > > I also had to add _PyMem_GetDefaultRawAllocator() to get a > deterministic memory allocator, rather than

Re: [Python-Dev] Python initialization and embedded Python

2017-11-21 16:57 GMT+01:00 Eric Snow : >> I understand that moving global variables to _PyRuntime helps to >> clarify how these variables are initialized and then finalized, but >> memory allocators are a complex corner case. > > Agreed. I spent a large portion of my

Re: [Python-Dev] Python initialization and embedded Python

On Mon, Nov 20, 2017 at 3:03 PM, Victor Stinner wrote: > To statically initialize PyMemAllocatorEx fields, you need to export a > lot of allocator functions. I would prefer to not do that. > > [snip] > > The rules to choose the allocator to each domain are also complex >

Re: [Python-Dev] Python initialization and embedded Python

On 21 November 2017 at 01:31, Eric Snow wrote: > On Nov 18, 2017 19:20, "Nick Coghlan" wrote: > > > OK, in that case I think the answer to Victor's question is: > > 1. Breaking calling Py_DecodeLocale() before calling Py_Initialize() > is a

Re: [Python-Dev] Python initialization and embedded Python

2017-11-20 22:35 GMT+01:00 Eric Snow : > I'm okay with that if we can't find another way. However, shouldn't > we be able to statically initialize the raw allocator in _PyRuntime, > much as we were doing before in obmalloc.c? I have a rough PR up: > >

Re: [Python-Dev] Python initialization and embedded Python

On Mon, Nov 20, 2017 at 8:43 AM, Victor Stinner wrote: > 2017-11-20 16:31 GMT+01:00 Eric Snow : >> That Py_DecodeLocale() can use PyMem_RawMalloc() pre-init is an >> implementation detail. > > Py_DecodeLocale() uses PyMem_RawMalloc(), and so

Re: [Python-Dev] Python initialization and embedded Python

2017-11-20 16:31 GMT+01:00 Eric Snow : > That Py_DecodeLocale() can use PyMem_RawMalloc() pre-init is an > implementation detail. Py_DecodeLocale() uses PyMem_RawMalloc(), and so its result must be freed by PyMem_RawFree(). It's part the documentation. I'm not sure

Re: [Python-Dev] Python initialization and embedded Python

On Nov 18, 2017 19:20, "Nick Coghlan" wrote: OK, in that case I think the answer to Victor's question is: 1. Breaking calling Py_DecodeLocale() before calling Py_Initialize() is a compatibility break with the API implied by our own usage examples, and we'll need to revert

Re: [Python-Dev] Python initialization and embedded Python

To not lost track of the issue, I created this issue on the bpo: https://bugs.python.org/issue32086 Victor 2017-11-20 7:54 GMT+01:00 Nick Coghlan : > On 19 November 2017 at 18:52, Victor Stinner wrote: >> Maybe we can find a compromise: revert the

Re: [Python-Dev] Python initialization and embedded Python

On 19 November 2017 at 18:52, Victor Stinner wrote: > Maybe we can find a compromise: revert the change on memory allocators. They > are too special to require to call PyRuntime_Init(). > > Currently, you cannot call PyMem_SetAllocators() before PyRuntime_Init(). At

Re: [Python-Dev] Python initialization and embedded Python

Maybe we can find a compromise: revert the change on memory allocators. They are too special to require to call PyRuntime_Init(). Currently, you cannot call PyMem_SetAllocators() before PyRuntime_Init(). Victor Le 19 nov. 2017 08:55, "Serhiy Storchaka" a écrit : >

Re: [Python-Dev] Python initialization and embedded Python

19.11.17 04:17, Nick Coghlan пише: 1. Breaking calling Py_DecodeLocale() before calling Py_Initialize() is a compatibility break with the API implied by our own usage examples, and we'll need to revert the breakage for 3.7, and ensure at least one release's worth of DeprecationWarning before

Re: [Python-Dev] Python initialization and embedded Python

On 19 November 2017 at 01:45, Serhiy Storchaka wrote: > 18.11.17 16:17, Nick Coghlan пише: >> >> On 18 November 2017 at 10:01, Victor Stinner >> wrote: >>> >>> I'm writing this email to ask if this change is an issue or not to >>> embedded Python

Re: [Python-Dev] Python initialization and embedded Python

18.11.17 16:17, Nick Coghlan пише: On 18 November 2017 at 10:01, Victor Stinner wrote: I'm writing this email to ask if this change is an issue or not to embedded Python and the Python C API. Is it still possible to call "all" functions of the C API before calling

Re: [Python-Dev] Python initialization and embedded Python

On 18 November 2017 at 10:01, Victor Stinner wrote: > I'm writing this email to ask if this change is an issue or not to > embedded Python and the Python C API. Is it still possible to call > "all" functions of the C API before calling Py_Initialize()? It isn't

Re: [Python-Dev] Python initialization and embedded Python

On Sat, 18 Nov 2017 01:01:47 +0100 Victor Stinner wrote: > > Maybe, the minimum change is to expose _PyRuntime_Initialize() in the > public C API? +1. Also a symmetric PyRuntime_Finalize() function (even if it's a no-op currently). Regards Antoine.

Re: [Python-Dev] Python initialization and embedded Python

18.11.17 02:01, Victor Stinner пише: Many global variables used by the "Python runtime" were move to a new single "_PyRuntime" variable (big structure made of sub-structures). See Include/internal/pystate.h. A side effect of moving variables from random files into header files is that it's not

Re: [Python-Dev] Python initialization and embedded Python

On 17Nov2017 1601, Victor Stinner wrote: In short, it means that using the "Python runtime" before it's initialized by _PyRuntime_Initialize() is now likely to crash. For example, calling PyMem_RawMalloc(), before calling _PyRuntime_Initialize(), now calls the function NULL: dereference a NULL