Re: [Python-Dev] PEP 575, 576, 579 and 580
On 2018-07-08 23:13, Mark Shannon wrote: I've added you suggestion, and everyone else's, to this github repo: https://github.com/markshannon/extended-calling-convention Feel free to comment on github, submit PRs or just email me directly if you have anything else you want to add. Do you agree to add this to PEP 579 instead? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
On 07/07/18 22:11, Jeroen Demeyer wrote: On 2018-07-07 15:38, Mark Shannon wrote: Hi, We seem to have a plethora of PEPs where we really ought to have one (or none?). - PEP 575 has been withdrawn. - PEP 579 is an informational PEP with the bigger picture; it does contain some of the requirements that you want to discuss here. - PEP 580 and PEP 576 are two alternative implementations of a protocol to optimize callables implemented in C. 5. It should speed up CPython for the standard benchmark suite. I'd like to replace this by: must *not slow down* the standard benchmark suite and preferable should not slow down anything. I've added you suggestion, and everyone else's, to this github repo: https://github.com/markshannon/extended-calling-convention Feel free to comment on github, submit PRs or just email me directly if you have anything else you want to add. Cheers, Mark. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
On Sat, Jul 7, 2018 at 6:38 AM, Mark Shannon wrote: > 1. The new API should be fully backwards compatible and shouldn't break the > ABI Which ABI? The stable ABI (PEP 384)? I don't think object layout is exposed there, though I'm not sure of the details. The regular ABI that almost everyone actually uses? That's already broken on every minor release, so you shouldn't spend any time worrying about preserving compatibility there. -n -- Nathaniel J. Smith -- https://vorpus.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
On Sat, Jul 7, 2018, 08:17 INADA Naoki, wrote: > > IMO, mailing lists are a terrible way to do software design, but a good > > way to gather requirements as it makes less likely that someone will be > > forgotten. > > > > Agreed. There are several topics we should discuss for these PEPs. > Mailing list is hard to follow. > > Can we have other communication channel? Dedicated Github repository? > zulip? or discuss.python.org? > > > So, let us gather the requirements for a new calling API. > > > > Here are my starting suggestions: > > > > 1. The new API should be fully backwards compatible and shouldn't break > > the ABI > > Agreed. We have chance to break ABI/API slightly at Python 4, although > breakage should be very small compared with Python 3. > > Until then, we should keep backward compatibility as possible. > > > 2. The new API should be used internally so that 3rd party extensions > > are not second class citizens in term of call performance. > > These PEPs proposes new public protocol which can be implemented > by 3rd party extensions, especially Cython. > In this meaning, it's not used only *internally*. > > > 3. The new API should not prevent 3rd party extensions having full > > introspection capabilities, supporting keyword arguments or another > > feature supported by Python functions. > > OK. > > > 4. The implementation should not exceed D lines of code delta and T > > lines of code in total size. I would suggest +200 and 1000 for D and T > > respectively (or is that too restrictive?). > > Hmm, I think this should be considered as (Frequency * Value) / > (Complexity). > Especially, if PEP 580 can removes 2000 lines of code, T>1000 seems OK. > I don't think any concrete number is really going to be helpful. This is probably going to come down to subjective "will this be complicated and hard to maintain?" And that call will probably come down to the BDFL for the PEP. -Brett 0p > > 5. It should speed up CPython for the standard benchmark suite. > > I think it's impossible in short term. We have specialized optimization > (FASTCALL and LOAD_METHOD/CALL_METHOD) already. > These optimization makes simple method calls 30% faster. > These PEPs makes 3rd party callable types can utilize these optimization. > > > 6. It should be understandable. > > > > OK. > While main audience is Cython, C extension writer should be able to use > new protocols by handwritten extension. > > > What am I missing? Comments from the maintainers of Cython and other > > similar tools would be appreciated. > > > > Cheers, > > Mark. > > > -- > INADA Naoki > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
On 2018-07-07 15:38, Mark Shannon wrote: Hi, We seem to have a plethora of PEPs where we really ought to have one (or none?). - PEP 575 has been withdrawn. - PEP 579 is an informational PEP with the bigger picture; it does contain some of the requirements that you want to discuss here. - PEP 580 and PEP 576 are two alternative implementations of a protocol to optimize callables implemented in C. 5. It should speed up CPython for the standard benchmark suite. I'd like to replace this by: must *not slow down* the standard benchmark suite and preferable should not slow down anything. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
> > > 5. It should speed up CPython for the standard benchmark suite. ... > > > > I don't think point 5 is a goal here either, as the problem isn't that > > these calling optimisations don't exist, it's that they don't > > currently have a public API that third party projects can access (the > > most recent METH_FASTCALL thread covers that pretty well). > > Agreed. The goal is not to speed up CPython but to bring third-party > extensions up to speed (both literally and figuratively). > For clarify, main goal is not just only 3rd party extension faster. Publicate some private APIs is enough for it. Goals of these PEP 576 (GitHub version) and 580 is making custom callable type (especially method-like object) faster. Because most functions and methods are defined with PyMethodDef and m_methods / tp_methods, these PEPs are not needed for them. I think main motivation of these PEPs are modern Python usages: Jupyter notebook + Cython. Unlike extension module writer, we shouldn't expect user knows difference between C and Python. That's why Cython want emulate normal Python function/methods as possible. Regards, -- INADA Naoki ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
INADA Naoki schrieb am 07.07.2018 um 17:16: >> 2. The new API should be used internally so that 3rd party extensions >> are not second class citizens in term of call performance. > > These PEPs proposes new public protocol which can be implemented > by 3rd party extensions, especially Cython. > In this meaning, it's not used only *internally*. I think Mark meant that the API should *also* be used internally, in the same way that external code uses it. Meaning, there shouldn't be a separate internal API. Stefan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
> IMO, mailing lists are a terrible way to do software design, but a good > way to gather requirements as it makes less likely that someone will be > forgotten. > Agreed. There are several topics we should discuss for these PEPs. Mailing list is hard to follow. Can we have other communication channel? Dedicated Github repository? zulip? or discuss.python.org? > So, let us gather the requirements for a new calling API. > > Here are my starting suggestions: > > 1. The new API should be fully backwards compatible and shouldn't break > the ABI Agreed. We have chance to break ABI/API slightly at Python 4, although breakage should be very small compared with Python 3. Until then, we should keep backward compatibility as possible. > 2. The new API should be used internally so that 3rd party extensions > are not second class citizens in term of call performance. These PEPs proposes new public protocol which can be implemented by 3rd party extensions, especially Cython. In this meaning, it's not used only *internally*. > 3. The new API should not prevent 3rd party extensions having full > introspection capabilities, supporting keyword arguments or another > feature supported by Python functions. OK. > 4. The implementation should not exceed D lines of code delta and T > lines of code in total size. I would suggest +200 and 1000 for D and T > respectively (or is that too restrictive?). Hmm, I think this should be considered as (Frequency * Value) / (Complexity). Especially, if PEP 580 can removes 2000 lines of code, T>1000 seems OK. > 5. It should speed up CPython for the standard benchmark suite. I think it's impossible in short term. We have specialized optimization (FASTCALL and LOAD_METHOD/CALL_METHOD) already. These optimization makes simple method calls 30% faster. These PEPs makes 3rd party callable types can utilize these optimization. > 6. It should be understandable. > OK. While main audience is Cython, C extension writer should be able to use new protocols by handwritten extension. > What am I missing? Comments from the maintainers of Cython and other > similar tools would be appreciated. > > Cheers, > Mark. -- INADA Naoki ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
On Sat, 7 Jul 2018 16:39:08 +0200 Stefan Behnel wrote: > Nick Coghlan schrieb am 07.07.2018 um 16:14: > > when the new calling > > convention is tied to a protocol that any type can implement (as PEP > > 580 proposes), the concern doesn't even arise. > > Nick, +1 to all of what you said in your reply, and I also really like the > fact that this proposal is creating a new, general protocol that removes > lots of type special casing from places where objects are being called > "efficiently". We're essentially designing a Fast Duck Calling convention > here. The Quick Quack protocole? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
On Sun, 8 Jul 2018 00:14:13 +1000 Nick Coghlan wrote: > > > So, let us gather the requirements for a new calling API. > > > Here are my starting suggestions: > > > > 1. The new API should be fully backwards compatible and shouldn't break the > > ABI > > 2. The new API should be used internally so that 3rd party extensions are > > not second class citizens in term of call performance. > > 3. The new API should not prevent 3rd party extensions having full > > introspection capabilities, supporting keyword arguments or another feature > > supported by Python functions. > > 4. The implementation should not exceed D lines of code delta and T lines of > > code in total size. I would suggest +200 and 1000 for D and T respectively > > (or is that too restrictive?). > > 5. It should speed up CPython for the standard benchmark suite. > > 6. It should be understandable. > > I like points 1, 2, 3, and 6, but I think point 4 should be a design > trade-off rather than a requirement, since minimising the delta in > CPython becomes an anti-goal if the outcome of doing so is to make the > change harder to adopt for third party projects (at the same time, a > delta that's too large is unlikely to be accepted, reviewed and > merged, which is what makes it a trade-off). > > I don't think point 5 is a goal here either, as the problem isn't that > these calling optimisations don't exist, it's that they don't > currently have a public API that third party projects can access (the > most recent METH_FASTCALL thread covers that pretty well). Agreed. The goal is not to speed up CPython but to bring third-party extensions up to speed (both literally and figuratively). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
Nick Coghlan schrieb am 07.07.2018 um 16:14: > when the new calling > convention is tied to a protocol that any type can implement (as PEP > 580 proposes), the concern doesn't even arise. Nick, +1 to all of what you said in your reply, and I also really like the fact that this proposal is creating a new, general protocol that removes lots of type special casing from places where objects are being called "efficiently". We're essentially designing a Fast Duck Calling convention here. Stefan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 575, 576, 579 and 580
On 7 July 2018 at 23:38, Mark Shannon wrote: > Hi, > > We seem to have a plethora of PEPs where we really ought to have one (or > none?). > > Traditionally when writing a new piece of software, one gathered > requirements before implementing the code. Let us return to that venerable > tradition. > > IMO, mailing lists are a terrible way to do software design, but a good way > to gather requirements as it makes less likely that someone will be > forgotten. That's the purpose of PEP 579: gather the background information on the problems that folks want to solve such that the competing proposed solutions aren't defining the problem that needs to be solved in different ways. If PEP 579 isn't working as a problem specification from your perspective, then I'd suggest posting a PR that Jeroen could review (although I think this thread is a good idea as well). > So, let us gather the requirements for a new calling API. > Here are my starting suggestions: > > 1. The new API should be fully backwards compatible and shouldn't break the > ABI > 2. The new API should be used internally so that 3rd party extensions are > not second class citizens in term of call performance. > 3. The new API should not prevent 3rd party extensions having full > introspection capabilities, supporting keyword arguments or another feature > supported by Python functions. > 4. The implementation should not exceed D lines of code delta and T lines of > code in total size. I would suggest +200 and 1000 for D and T respectively > (or is that too restrictive?). > 5. It should speed up CPython for the standard benchmark suite. > 6. It should be understandable. I like points 1, 2, 3, and 6, but I think point 4 should be a design trade-off rather than a requirement, since minimising the delta in CPython becomes an anti-goal if the outcome of doing so is to make the change harder to adopt for third party projects (at the same time, a delta that's too large is unlikely to be accepted, reviewed and merged, which is what makes it a trade-off). I don't think point 5 is a goal here either, as the problem isn't that these calling optimisations don't exist, it's that they don't currently have a public API that third party projects can access (the most recent METH_FASTCALL thread covers that pretty well). My own additional concern that I think is also on the debatable border between "design requirement" and "design trade-off" is whether or not it's acceptable for us to require that existing third party projects change their parent CPython type in order to access the optimised calling conventions. Changing Python base types in an extension module can end up being an annoyingly intrusive change, since it changes the memory layout in your instances. Whether or not that's a problem depends on exactly what you're doing, but when the new calling convention is tied to a protocol that any type can implement (as PEP 580 proposes), the concern doesn't even arise. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
