[Python-Dev] Currently working on the release of Python 3.8.0rc1
Team, amazing job on getting us back on track over the weekend. Thank you All release blockers and deferred release blockers solved. And there was relatively little additional activity on the branch -- as expected at this point! Thank you for this, it will help get the release candidate out on time. I'm working on cutting RC1 today Hopefully all sanity checks, as well as building the source tarball and the binary installers for macOS and Windows will all work out fine and we'll be seeing RC1 out tonight. RC2 and the date of 3.8.0 gold If we manage to avoid the need for RC2, we will be able to release 3.8.0 on October 14th. If we need an RC2, that will slip by a week. I hope we won't. Ideally RC1 should be identical codewise to 3.8.0. To help our chances, please avoid any source-related activity on the 3.8 branch from now until the release of 3.8.0. Yes, that also counts for bug fixes unless they are critical. (Yeah, it might be a bit annoying but nobody wants to be the person who introduced a last minute regression in a major release.) Note I didn't say I forbid any activity. I have no power over you, actually. More importantly though, I trust your judgement if you assess some bug is bad enough the fix absolutely has to get into 3.8.0. Moreover, I specifically said source-related activity because... 3.8.0 is important. To some users it will be the first and the last release in the 3.8 series they will see. We need you to focus on the docs now Are all your changes properly documented? Did you notice other changes you know of to have insufficient documentation? Can you help with the "What's New" document? anxiously configure-&&-makingly y'rs - Ł signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/ACMKEQNGK4FVUIZ6TYD5H26OSPIO5GSN/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
On 2019-09-30 00:33, Nick Coghlan wrote: [...] We'll never change the size of the config structs (or any other public struct) in a maintenance branch But we *will* change it in alphas/betas, and then we will ask people to try their software out with these. Of course, there are no actual API/ABI guarantees with alphas/betas: officially, you need to rebuild everything for each new pre-release. But with the goal being to find bugs rather than provide correct software, it's OK to take shortcuts (if you can then distinguish the bugs the shortcuts cause from actual bugs in the code). Issuing proper version mismatch errors (instead of random cryptic segfaults) could make for a much nicer experience when testing several successive alphas/betas. ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/LEXOES2PE634F5JJQJGXFTFATX3CY3PA/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
On Mon, 30 Sep 2019 at 09:37, Victor Stinner wrote:
>
> Le lun. 30 sept. 2019 à 00:33, Nick Coghlan a écrit :
> > As noted above, despite what I wrote on BPO, you no longer need to persuade
> > me that the version check is desirable, only that a narrow check on
> > specific struct sizes is preferable to a broad check on the expected API
> > version.
>
> I understand that your main motivation to use the Python version
> number rather than sizeof(PyConfig) is the error message.
No, my main motivation is to create an API that can emit a useful
error message on *ALL* version conflicts between an embedding
application and the embedded runtime, not just version conflicts
involving versions that change the size of the config structs.
The latter option is a good one if all we want to version is the
struct itself, but I don't think that's what we really want here: I
think we want to version check the entire API/ABI.
The improved error message is just a nice bonus, as is the fact that I
think the following code is more self-explanatory than the size
checks:
PyConfig config;
config.header_version = PY_VERSION_HEX;
PyStatus status = PyConfig_InitPythonConfig(&config);
The documentation would say something like "header_version should
specify the version of the CPython headers that was used to build and
link the embedding application. The simplest way to set it correctly
is to set it to ``PY_VERSION_HEX``".
> If we implement support for older PyConfig ("stable ABI"), you will
> simply never see this error: it will just work transparently.
But that's true regardless of whether the consistency check is based
on the struct size or the header version.
> IMHO the current error message is good enough:
>
> if (config->struct_size != sizeof(PyConfig)) {
> return _PyStatus_ERR("unsupported PyConfig structure size "
> "(Python version mismatch?)");
> }
That error message won't trigger if the public API struct that changed
size is PyTypeObject or PyObject (or any of the other structs in the
full API/ABI), whereas a PY_VERSION_HEX based check can be made to
fail based on any policy we choose to enforce.
For example, we could make the runtime enforce the following pair of rules:
* if both the header version and runtime version are final releases or
release candidates, then only the major and minor version fields need
to match (the X.Y in X.Y.Z)
* if either the header version or the runtime version is an alpha or
beta release (or an unknown release type), then the entire version
number must match exactly
Or we could be more permissive, and only ever enforce the first rule,
even for alpha and beta releases.
> I wrote a proof-of-concept to check if it would be doable to support
> multiple versions (sizes) of PyConfig: it's doable and it's quite easy
> to implement, a few lines of code. For example, support Python 3.8
> PyConfig in Python 3.9.
That doesn't change all that much with a Py_VERSION_HEX based check -
it just needs an extra table for range based lookups to find the
relevant struct size for the given version, together with keeping
definitions of the old config structs around.
typedef struct {
uint32_t firstVersion;
Py_ssize_t structSize;
} _Py_versionToSize;
static _Py_versionToSize _PyConfigStructSizes {
{0x0309, sizeof(_PyConfig_3_9)},
{0, sizeof(_PyConfig_3_8)}
}
If the given version matched, you wouldn't need to do the size lookup.
You also wouldn't be constrained to having the lookup table be from
versions to struct sizes - it could instead be from versions to
function pointers, or some other such thing, based on whatever made
the most sense given how the rest of the config system worked in any
given release.
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/RL2TEQ6BEGOHSWPPUVML5SLNIBVGFJ5G/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
Le lun. 30 sept. 2019 à 13:45, Nick Coghlan a écrit :
> > I understand that your main motivation to use the Python version
> > number rather than sizeof(PyConfig) is the error message.
>
> No, my main motivation is to create an API that can emit a useful
> error message on *ALL* version conflicts between an embedding
> application and the embedded runtime, not just version conflicts
> involving versions that change the size of the config structs.
>
> The latter option is a good one if all we want to version is the
> struct itself, but I don't think that's what we really want here: I
> think we want to version check the entire API/ABI.
IMHO this discussion is going too far away from the PEP 587 goals. The
PEP 587 is unrelated to the stable API or the stable ABI.
I suggest to develop a new use case and a new solution to handle this
case. The solution can be *very different* than "compare two versions
at startup".
For example, we may need a CI to ensure that we don't break the stable ABI:
https://pythoncapi.readthedocs.io/stable_abi.html#check-for-abi-changes
Another solution is to write a completely new C API from scratch. For
example, HPy ("PyHandle") project proposes to write a new API
implemented on top of the existing API:
https://github.com/pyhandle/hpy
This project is even developed outside CPython. So the solution can be
found *outside* CPython as well.
Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/UTAGQ3GIP3OZBLE4LWCI2GVDCDAVRPTF/
[Python-Dev] Re: [python-committers] Currently working on the release of Python 3.8.0rc1
On Mon, 30 Sep 2019 at 17:48, Łukasz Langa wrote: > > Team, > amazing job on getting us back on track over the weekend. > > Thank you > All release blockers and deferred release blockers solved. And there was > relatively little additional activity on the branch -- as expected at this > point! Thank you for this, it will help get the release candidate out on time. > > I'm working on cutting RC1 today > Hopefully all sanity checks, as well as building the source tarball and the > binary installers for macOS and Windows will all work out fine and we'll be > seeing RC1 out tonight. I've filed https://bugs.python.org/issue38326 as a release blocker, as I don't think we should be cutting RCs when changes have been made to a PEP-approved API without any pre-merge design discussion. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/UXZJ6BKHTHNPNHCSSFN6BM75SELGUPOG/
[Python-Dev] Re: [python-committers] Currently working on the release of Python 3.8.0rc1
> On 30 Sep 2019, at 16:09, Nick Coghlan wrote: > > I've filed https://bugs.python.org/issue38326 as a release blocker, as > I don't think we should be cutting RCs when changes have been made to > a PEP-approved API without any pre-merge design discussion. Nick, Victor, as co-authors of said PEP, please reach a decision ASAP. - Ł signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/BB6Q6WVKJC6F7SX4XVVJIFS2I7NFI322/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
I agree that passing a struct size as struct member values sounds a bit unidiomatic. Also, it doesn't achieve ABI stability, just allows erroring out in case the user links with the wrong Python version. Regards Antoine. On Sun, 29 Sep 2019 16:47:41 +1000 Nick Coghlan wrote: > On Sat, 28 Sep 2019 at 12:56, Victor Stinner wrote: > > > > Hi, > > > > I dislike having to do that, but I had to make a last minute change in > > my PEP 587 "Python Initialization Configuration" to allow to modify > > the structure in the future without breaking the backward > > compatibility. I added a "struct_size" field to PyPreConfig and > > PyConfig: > > > > * https://bugs.python.org/issue38304 > > * > > https://github.com/python/peps/commit/afa38c0bef7738b8fcc3173daee8488e0420833d > > > > The example: > > > >PyConfig config; > >PyStatus status = PyConfig_InitPythonConfig(&config); > >... > > > > must now be written: > > > >PyConfig config; > >config.struct_size = sizeof(PyConfig); > >PyStatus status = PyConfig_InitPythonConfig(&config); > >... > > > > At the beginning, I used a private "_config_version" field which was > > initialized statically by a macro. But it was decided to replace > > macros with functions. I only noticed today that the conversion to > > function broke the API/ABI future compatibility. > > > > PyConfig_InitPythonConfig() got uninitialized memory and didn't know > > the size of the config variable. > > I don't quite understand the purpose of this change, as there's no > stable ABI for applications embedding CPython. As a result, updating > to a new X.Y.0 release always requires rebuilding the entire > application, not just building and relinking CPython. > > I could understand a change to require passing in an expected Python > version so we can fail more gracefully on a bad link where an > application that intended to embed Python 3.8 is incorrectly linked > against Python 3.9 (for example), but performing that kind of check > would require passing in PY_VERSION_HEX, not the size of the config > struct. > > > With my change, the function now requires the "struct_size" field to > > be set, and so it can support internally different versions of the > > PyConfig structure. > > > > Storing the structure size directly in the structure is a common > > practice in the Windows API which is a good example of long term ABI > > compatibility. > > This analogy doesn't hold, as Microsoft explicitly support running old > binaries on new versions of Windows, and hence need a way to determine > which version of the structure is being passed in. > > We don't support that - all our APIs that accept > PyObject/PyTypeObject/etc require that the caller pass in structs of > the correct size for the version of Python being used. The PyConfig > and PyPreConfig structs are no different from PyObject in that regard: > if there's a size mismatch, then the developers of the embedding > application have somehow messed up their build process. > > The stable ABI is a different story, but that's why we try very hard > to avoid exposing any structs in the stable ABI. > > Cheers, > Nick. > ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/ZDZ7DFZZIUYGQJOGZ3ONTTCRYGZ6KVCS/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
On Mon, 30 Sep 2019 at 23:50, Victor Stinner wrote:
>
> Le lun. 30 sept. 2019 à 13:45, Nick Coghlan a écrit :
> > > I understand that your main motivation to use the Python version
> > > number rather than sizeof(PyConfig) is the error message.
> >
> > No, my main motivation is to create an API that can emit a useful
> > error message on *ALL* version conflicts between an embedding
> > application and the embedded runtime, not just version conflicts
> > involving versions that change the size of the config structs.
> >
> > The latter option is a good one if all we want to version is the
> > struct itself, but I don't think that's what we really want here: I
> > think we want to version check the entire API/ABI.
>
> IMHO this discussion is going too far away from the PEP 587 goals. The
> PEP 587 is unrelated to the stable API or the stable ABI.
I would be entirely happy with reverting to PEP 587 as originally
accepted (no public struct_size field, no hidden struct versioning
field), rather than making last minute changes to it right before the
first release candidate.
PyConfig and PyPreConfig aren't covered by the stable ABI, so there's
no current need for them to be able to evolve gracefully across ABI
version bumps. If we *do* decide to evolve them in a direction that
allows for more explicit ABI version checks, then doing that as a
separate PyVersionedConfig and PyVersionedPreConfig API would have the
virtue of keeping the simple case simple, while offering a slightly
more awkward to use, but also more forward compatible API for the
benefits of folks that wanted to use it.
However, *if* we were to add a versioning scheme for 3.8.0rc1, then
https://github.com/python/cpython/pull/16496/files covers what I think
it should look like (a header_version field that gets set to
PY_VERSION_HEX by the embedding application).
The only outcome I'd consider undesirable is shipping a public API
that's more awkward to use than it needs to be, doesn't conform to the
accepted version of the PEP, and doesn't protect against most of the
potential sources of segfaults arising from an ABI mismatch (and could
even cause them if the supplied "struct_size" value were to be trusted
directly, rather than being treated as a lookup value into a set of
known-valid struct sizes. The currently checked in code at least
doesn't do that, but that's only because it ignores the new
struct_size field entirely when making the memset() calls to
initialise the config structs).
> I suggest to develop a new use case and a new solution to handle this
> case. The solution can be *very different* than "compare two versions
> at startup".
>
> For example, we may need a CI to ensure that we don't break the stable ABI:
> https://pythoncapi.readthedocs.io/stable_abi.html#check-for-abi-changes
>
> Another solution is to write a completely new C API from scratch. For
> example, HPy ("PyHandle") project proposes to write a new API
> implemented on top of the existing API:
> https://github.com/pyhandle/hpy
>
> This project is even developed outside CPython. So the solution can be
> found *outside* CPython as well.
These are far outside the bounds of "Don't segfault when an embedding
application tries to do something that's obviously broken", which I
consider the only reason to even consider changing the public API away
from what was accepted in PEP 587 in the 3.8rc1 time frame.
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/UXRKYP3YCAY2K2GD3QLCGZGQ45WMKORZ/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
On Tue., 1 Oct. 2019, 3:05 am Nick Coghlan, wrote: > > The only outcome I'd consider undesirable is shipping a public API > that's more awkward to use than it needs to be, doesn't conform to the > accepted version of the PEP, and doesn't protect against most of the > potential sources of segfaults arising from an ABI mismatch (and could > even cause them if the supplied "struct_size" value were to be trusted > directly, rather than being treated as a lookup value into a set of > known-valid struct sizes. The currently checked in code at least > doesn't do that, but that's only because it ignores the new > struct_size field entirely when making the memset() calls to > initialise the config structs). > My apologies, the "only" here is incorrect, as the existing code also has guards to make sure that the passed in struct size is exactly the same as the interpreter's compile time size (it's effectively a lookup set with one member). Cheers, Nick. > ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/IHRN6EFQ3OVPCJVDUIQRU4GVKUA3SYQC/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
On Mon., 30 Sep. 2019, 7:43 pm Petr Viktorin, wrote: > On 2019-09-30 00:33, Nick Coghlan wrote: > [...] > > We'll never change the size of the config structs (or any other public > > struct) in a maintenance branch > > But we *will* change it in alphas/betas, and then we will ask people to > try their software out with these. > Of course, there are no actual API/ABI guarantees with alphas/betas: > officially, you need to rebuild everything for each new pre-release. But > with the goal being to find bugs rather than provide correct software, > it's OK to take shortcuts (if you can then distinguish the bugs the > shortcuts cause from actual bugs in the code). > > Issuing proper version mismatch errors (instead of random cryptic > segfaults) could make for a much nicer experience when testing several > successive alphas/betas. > That's actually the variant I implemented in my PR (release candidates and final releases allow nominally compatible inexact matches, everything else requires an exact version match). Cheers, Nick. ___ > Python-Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/LEXOES2PE634F5JJQJGXFTFATX3CY3PA/ > ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/WAHKL7GVBGFNGTO25BVJLDTRLUDLKDXX/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
Hi back, It seems like "config.struct_size = sizeof(PyConfig);" syntax is "not really liked". I don't see any traction to provide a stable ABI for embedded Python. The consensus is more towards: "it doesn't work and we don't want to bother with false promises" (nor add code for that). Since Lukasz is supposed to tag 3.8.0rc1 today, I propose to remove anything related to "stable API" or "stable ABI" from the PEP 587 right now. I wrote two PRs to remove struct_size: * CPython: https://github.com/python/cpython/pull/16500 * PEP 587: https://github.com/python/peps/pull/1185 Later, if we decide to start proving a stable ABI for embedded Python, we can still add a "version" or "struct_size" field to PyConfig later (for example in Python 3.9). Victor ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/2JAJQA5OANFPXAJ3327RRPHPQLKVP2EW/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
Victor Stinner wrote: > Hi Nick, > Le dim. 29 sept. 2019 à 08:47, Nick Coghlan [email protected] a écrit : > > I don't quite understand the purpose of this change, > > as there's no > > stable ABI for applications embedding CPython. > > Well, I would like to prepare Python to provide a stable ABI for > embedded Python. While it's not a design goal yet > (Include/cpython/initconfig.h is currently excluded from > Py_LIMITED_API), this change is a step towards that. So then isn't this a very last-minute, premature optimization if it isn't a design goal yet? If that's the case then I would vote not to make the change and wait until there's been feedback on 3.8 and then look at stabilizing the embedding API such that it can be considered stable. -Brett > > As a result, updating > > to a new X.Y.0 release always requires rebuilding the entire > > application, not just building and relinking CPython. > > In Python 3.8, C extensions are no longer linked to libpython which > allows to switch between a release build and a debug build of > libpython. > Can we imagine the same idea for embedded Python? I checked vim on > Linux: it's linked to libpython3.7m.so.1.0: a specific Python version, > library built in release mode. > > I could understand a change to require passing in an > > expected Python > > version so we can fail more gracefully on a bad link where an > > application that intended to embed Python 3.8 is incorrectly linked > > against Python 3.9 (for example), but performing that kind of check > > would require passing in PY_VERSION_HEX, not the size of the config > > struct. > > It seems simpler to me to pass the structure size rather than the > Python version. It avoids the risk of updating the structure without > update the Python version. I also avoids to have to change the Python > version immediately when PyConfig is modified. The main risk of > sizeof(PyConfig) comes if we remove a field and add a new field of > the same size: the structure size doesn't change... But in my > experience, we only add new ways to configure Pyhon, we never remove > old ones :-D > The question is if it's convenient to compute sizeof(PyConfig) in > programming languages other than C. Providing a "structure version" or > the structure size from a function call would not work. The value must > be known a compilation time, not at runtime. The purpose is to compare > the version/size between build and runtime (they must match). > In the first implementation of my PEP, I used an internal "config > version" provides by a macro. But it was said that macros are not > convenient. > PY_VERSION_HEX is provided as a macro, but we are now trying to avoid > macros in our C API, no? At least, it's what I understood from the PEP > 587 discussion. > > We don't support that - all our APIs that accept > > PyObject/PyTypeObject/etc require that the caller pass in structs of > > the correct size for the version of Python being used. > > For PyConfig, it's allocated (on the stack on or on heap) by the > application. So the application requires to access to the full > structure. > Objects (instances) are allocated by Python (on the heap). > Applications usually don't need to know/access the structure. > Python is far from being perfect, static types are still supported and > they are an issue for the stable ABI. > > The PyConfig > > and PyPreConfig structs are no different from PyObject in that regard: > > if there's a size mismatch, then the developers of the embedding > > application have somehow messed up their build process. > > In short, PyConfig initialization works like that: > > The application allocates a PyConfig object on the stack > Python calls memset(config, 0, sizeof(PyConfig)) > > If there is a size mismatch, Python triggers a buffer overflow which > is likely to cause issues. > I prefer to have a clean API which makes buffer overflow impossible. > Embedding Python and handling different Python versions is not > trivial, especially if we start to add new fields to each PyConfig > (which is very likely). If prefer to be extra careful. > I also expect bad surprises even in CPython with Programs/_testembed: > many tests use PyConfig. Depending if _testembed is properly rebuilt > or not, bad thing will happen. > -- > To implement my PEP 445 "Add new APIs to customize Python memory > allocators", I added a PyMemAllocator structure which is part of the > API. > Quickly, I had to add a new field to PyMemAllocator. But it wasn't > possible to detect if a C extension used the old or the new > structure... So I decided to rename the structure to PyMemAllocatorEx > to ensure that the compilation of all C extensions using the API will > fail... :-( > I really dislike this solution. What will happen when we will add > another field to the structure, like a new PyMem_Aligned() (similar to > posix_memalign()) function? PyMem_Aligned() can be implementation on > top of an existing memory allocator which doesn't support it natively. > B
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
On Mon, 30 Sep 2019 22:43:40 +0200 Victor Stinner wrote: > Hi back, > > It seems like "config.struct_size = sizeof(PyConfig);" syntax is "not > really liked". > > I don't see any traction to provide a stable ABI for embedded Python. Not at the last minute in a rc1, I'd say :-) I think if you wanted to make the PyConfig apt at providing ABI-stability, you should have designed it differently. `PyType_FromSpec` provides a useful model (pass an arbitrary-sized static array of field initializers). Regards Antoine. ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/FJ2CT5ISRMH4PGI22VTU3JSUJ7AFSYLE/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
Le lun. 30 sept. 2019 à 23:26, Antoine Pitrou a écrit : > I think if you wanted to make the PyConfig apt at providing > ABI-stability, you should have designed it differently. > `PyType_FromSpec` provides a useful model (pass an arbitrary-sized > static array of field initializers). PyConfig is even more complex than PyTypeObject :-/ The following PyConfig methods allocates memory on the heap using PyMem_RawMalloc(): PyConfig_SetArgv() PyConfig_SetBytesArgv() PyConfig_SetBytesString() PyConfig_SetString() PyConfig_SetWideStringList() Setting a field can fail with a memory allocation failure, preinitialization error, or other errors. But right, I get your point, there are other solutions. Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/KW367IBINBG7OWVOVU4DBWJLZDIII27J/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
On Tue., 1 Oct. 2019, 6:47 am Victor Stinner, wrote: > Hi back, > > It seems like "config.struct_size = sizeof(PyConfig);" syntax is "not > really liked". > > I don't see any traction to provide a stable ABI for embedded Python. > The consensus is more towards: "it doesn't work and we don't want to > bother with false promises" (nor add code for that). > > Since Lukasz is supposed to tag 3.8.0rc1 today, I propose to remove > anything related to "stable API" or "stable ABI" from the PEP 587 > right now. I wrote two PRs to remove struct_size: > > * CPython: https://github.com/python/cpython/pull/16500 > * PEP 587: https://github.com/python/peps/pull/1185 > > Later, if we decide to start proving a stable ABI for embedded Python, > we can still add a "version" or "struct_size" field to PyConfig later > (for example in Python 3.9). > Thanks Victor, I think this is the right way for us to go, given the relative timing in the release cycle. The idea of some day expanding the stable ABI to cover embedding applications, not just extension modules, is an intriguing one, but I'm genuinely unsure anyone would ever actually use it. Cheers, Nick. > Victor > ___ > Python-Dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/2JAJQA5OANFPXAJ3327RRPHPQLKVP2EW/ > ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/OMPEQEEJEKUH2KVSUV5UDGMI7IP3PP4Y/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
On Tue., 1 Oct. 2019, 8:38 am Nick Coghlan, wrote: > Later, if we decide to start proving a stable ABI for embedded Python, >> we can still add a "version" or "struct_size" field to PyConfig later >> (for example in Python 3.9). >> > > Thanks Victor, I think this is the right way for us to go, given the > relative timing in the release cycle. > > The idea of some day expanding the stable ABI to cover embedding > applications, not just extension modules, is an intriguing one, but I'm > genuinely unsure anyone would ever actually use it. > After merging your PR and closing mine, I had an idea for Python 3.9: what if we offered a separate public "int Py_CheckVersionCompatibility(uint64_t header_version)" call? (64 bit input rather than 32 to allow for possible future changes to the version number formatting scheme) The basic variant of that API would be what I had in my PR: release candidates and final releases allow an inexact match, other releases require the hex version to match exactly. If an embedding application called that before calling any PreConfig or Config APIs, they'd get the same easier debugging for version mismatches, without needing to change the config structs themselves. Instead, we'd only need a new PreConfig field if we wanted to start having other runtime APIs change their behaviour based on the active nominal Python version. As an added bonus, extension modules could also optionally call that compatibility checking API early in their init function. If we wanted to get more exotic with the internal design, we could maintain a table of beta versions containing known ABI *breaks* (e.g. public structs changing size), and also permit inexact matches for beta releases, as long as the given header version was newer than the last ABI break, but older than the runtime version. That table could be reset to empty when the ABI was frozen for a release series (thus causing a merge conflict if a backport was requested for a patch that changed that table). Cheers, Nick. >> ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/7A7HYAAQWBZEEFRO4PCA5ARCX6U2DTVV/
[Python-Dev] Re: PEP 587 (Python Initialization Configuration) updated to be future proof again
On 30Sep2019 1625, Nick Coghlan wrote: After merging your PR and closing mine, I had an idea for Python 3.9: what if we offered a separate public "int Py_CheckVersionCompatibility(uint64_t header_version)" call? (64 bit input rather than 32 to allow for possible future changes to the version number formatting scheme) The basic variant of that API would be what I had in my PR: release candidates and final releases allow an inexact match, other releases require the hex version to match exactly. As I posted on the issue tracker, I don't think this kind of version verification is necessary unless we're genuinely going to get into the business of adding built-in shims for API changes. Every supported platform is going to make you link against a dynamic library with the version number in its name. (Every approach that avoids this is not supported or does not support embedding.) So you get run-time checking for free, and you can add a compile-time check in three or four different ways depending on what makes the most sense for your build system. Let's not complicate the embedding API further with unnecessary new APIs. I'm keen to design something broadly useful and simpler than what we have now, and this is not going to help with it. Cheers, Steve ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/35WNLKQL4DWZZ6MS7OEZEDV7AM2IZFKV/
[Python-Dev] Re: [python-committers] Currently working on the release of Python 3.8.0rc1
Hello Łukasz, I consider this one critical enough to get into 3.8: https://bugs.python.org/issue38319 Long story short: shutil.copyfile() and socket.sendfile() are broken on 32-bit platforms for files >= 2GiB. shutil.copyfile() was modified by me in the 3.8 cycle so the bug only affects 3.8 and 3.9. On Mon, Sep 30, 2019 at 3:48 PM Łukasz Langa wrote: > Team, > amazing job on getting us back on track over the weekend. > > *Thank you* > All release blockers and deferred release blockers solved. And there was > relatively little additional activity on the branch -- as expected at this > point! Thank you for this, it will help get the release candidate out on > time. > > *I'm working on cutting RC1 today* > Hopefully all sanity checks, as well as building the source tarball and > the binary installers for macOS and Windows will all work out fine and > we'll be seeing RC1 out tonight. > > *RC2 and the date of 3.8.0 gold* > If we manage to avoid the need for RC2, we will be able to release 3.8.0 > on October 14th. If we need an RC2, that will slip by a week. I hope we > won't. Ideally RC1 should be identical codewise to 3.8.0. > > To help our chances, please avoid any source-related activity on the 3.8 > branch from now until the release of 3.8.0. Yes, that also counts for bug > fixes unless they are critical. (Yeah, it might be a bit annoying but > nobody wants to be the person who introduced a last minute regression in a > major release.) > > Note I didn't say I forbid any activity. I have no power over you, > actually. More importantly though, I trust your judgement if you assess > some bug is bad enough the fix absolutely has to get into 3.8.0. Moreover, > I specifically said source-related activity because... > > 3.8.0 is important. To some users it will be the first and the last > release in the 3.8 series they will see. > > *We need you to focus on the docs now* > Are all your changes properly documented? > Did you notice other changes you know of to have insufficient > documentation? > Can you help with the "What's New" document? > > anxiously configure-&&-makingly y'rs > - Ł > ___ > python-committers mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3/lists/python-committers.python.org/ > Message archived at > https://mail.python.org/archives/list/[email protected]/message/ACMKEQNGK4FVUIZ6TYD5H26OSPIO5GSN/ > Code of Conduct: https://www.python.org/psf/codeofconduct/ > -- Giampaolo - http://grodola.blogspot.com ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/VC7XNBK65YBNUHKXC7PNMJQCU5P5YJ4X/
