Hi,
On 02/04/2019 1:49 pm, Petr Viktorin wrote:
On 3/30/19 11:36 PM, Jeroen Demeyer wrote:
On 2019-03-30 17:30, Mark Shannon wrote:
2. The claim that PEP 580 allows "certain optimizations because other
code can make assumptions" is flawed. In general, the caller cannot make
assumptions about the callee or vice-versa. Python is a dynamic
language.
PEP 580 is meant for extension classes, not Python classes. Extension
classes are not dynamic. When you implement tp_call in a given way,
the user cannot change it. So if a class implements the C call
protocol or the vectorcall protocol, callers can make assumptions
about what that means.
PEP 579 is mainly a list of supposed flaws with the
'builtin_function_or_method' class.
The general thrust of PEP 579 seems to be that builtin-functions and
builtin-methods should be more flexible and extensible than they are. I
don't agree. If you want different behaviour, then use a different
object. Don't try an cram all this extra behaviour into a pre-existing
object.
I think that there is a misunderstanding here. I fully agree with the
"use a different object" solution. This isn't a new solution: it's
already possible to implement those different objects (Cython does
it). It's just that this solution comes at a performance cost and
that's what we want to avoid.
It does seem like there is some misunderstanding.
PEP 580 defines a CCall structure, which includes the function pointer,
flags, "self" and "parent". Like the current implementation, it has
various METH_ flags for various C signatures. When called, the info from
CCall is matched up (in relatively complex ways) to what the C function
expects.
PEP 590 only adds the "vectorcall". It does away with flags and only has
one C signatures, which is designed to fit all the existing ones, and is
well optimized. Storing the "self"/"parent", and making sure they're
passed to the C function is the responsibility of the callable object.
There's an optimization for "self" (offsetting using
PY_VECTORCALL_ARGUMENTS_OFFSET), and any supporting info can be provided
as part of "self". >
I'll reiterate that PEP 590 is more general than PEP 580 and that once
the callable's code has access to the callable object (as both PEPs
allow) then anything is possible. You can't can get more extensible than
that.
Anything is possible, but if one of the possibilities becomes common and
useful, PEP 590 would make it hard to optimize for it.
Python has grown many "METH_*" signatures over the years as we found
more things that need to be passed to callables. Why would
"METH_VECTORCALL" be the last? If it won't (if you think about it as one
more way to call functions), then dedicating a tp_* slot to it sounds
quite expensive.
I doubt METH_VECTORCALL will be the last.
Let me give you an example: It is quite common for a function to take
two arguments, so we might want add a METH_OO flag for builtin-functions
with 2 parameters.
To support this in PEP 590, you would make exactly the same change as
you would now; which is to add another case to the switch statement in
_PyCFunction_FastCallKeywords.
For PEP 580, you would add another case to the switch in PyCCall_FastCall.
No difference really.
PEP 580 uses a slot as well. It's only 8 bytes per class.
In one of the ways to call C functions in PEP 580, the function gets
access to:
- the arguments,
- "self", the object
- the class that the method was found in (which is not necessarily
type(self))
I still have to read the details, but when combined with
LOAD_METHOD/CALL_METHOD optimization (avoiding creation of a "bound
method" object), it seems impossible to do this efficiently with just
the callable's code and callable's object.
It is possible, and relatively straightforward.
Why do you think it is impossible?
I would argue the opposite: PEP 590 defines a fixed protocol that is
not easy to extend. PEP 580 on the other hand uses a new data
structure PyCCallDef which could easily be extended in the future
(this will intentionally never be part of the stable ABI, so we can do
that).
I have also argued before that the generality of PEP 590 is a bad
thing rather than a good thing: by defining a more rigid protocol as
in PEP 580, more optimizations are possible.
PEP 580 has the same limitation for the same reasons. The limitation is
necessary for correctness if an object supports calls via `__call__` and
through another calling convention.
I don't think that this limitation is needed in either PEP. As I
explained at the top of this email, it can easily be solved by not
using the protocol for Python classes. What is wrong with my proposal
in PEP 580: https://www.python.org/dev/peps/pep-0580/#inheritance
I'll add Jeroen's notes from the review of the proposed PEP 590
(https://github.com/python/peps/pull/960):
The statement "PEP 580 is specifically targetted at function-like
objects, and doesn't support other callables like classes, partial
functions, or proxies" is factually false. The motivation for PEP 580 is
certainly function/method-like objects but it's a general protocol that
every class can implement. For certain classes, it may not be easy or
desirable to do that but it's always possible. >
Given that `PY_METHOD_DESCRIPTOR` is a flag for tp_flags, shouldn't it
be called `Py_TPFLAGS_METHOD_DESCRIPTOR` or something?
Py_TPFLAGS_HAVE_VECTOR_CALL should be Py_TPFLAGS_HAVE_VECTORCALL, to be
consistent with tp_vectorcall_offset and other uses of "vectorcall" (not
"vector call")
Thanks for the comments, I'll update the PEP when I get the chance.
And mine, so far:
I'm not clear on the constness of the "args" array.
If it is mutable (PyObject **), you can't, for example, directly pass a
tuple's storage (or any other array that could be used in the call).
If it is not (PyObject * const *), you can't insert the "self" argument in.
The reference implementations seems to be inconsistent here. What's the
intention?
I'll make it clearer in the PEP.
My thinking was that if `PY_VECTORCALL_ARGUMENTS_OFFSET` is set then the
caller is allowing the callee to mutate element -1.
It would make sense to generalise that to any element of the vector
(including -1).
When passing the contents of a tuple, `PY_VECTORCALL_ARGUMENTS_OFFSET`
should not be set, and thus the vector could not be mutated.
Cheers,
Mark.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com