On 3/30/19 11:36 PM, Jeroen Demeyer wrote:
On 2019-03-30 17:30, Mark Shannon wrote:
2. The claim that PEP 580 allows "certain optimizations because other
code can make assumptions" is flawed. In general, the caller cannot make
assumptions about the callee or vice-versa. Python is a dynamic language.
PEP 580 is meant for extension classes, not Python classes. Extension
classes are not dynamic. When you implement tp_call in a given way, the
user cannot change it. So if a class implements the C call protocol or
the vectorcall protocol, callers can make assumptions about what that
means.
PEP 579 is mainly a list of supposed flaws with the
'builtin_function_or_method' class.
The general thrust of PEP 579 seems to be that builtin-functions and
builtin-methods should be more flexible and extensible than they are. I
don't agree. If you want different behaviour, then use a different
object. Don't try an cram all this extra behaviour into a pre-existing
object.
I think that there is a misunderstanding here. I fully agree with the
"use a different object" solution. This isn't a new solution: it's
already possible to implement those different objects (Cython does it).
It's just that this solution comes at a performance cost and that's what
we want to avoid.
It does seem like there is some misunderstanding.
PEP 580 defines a CCall structure, which includes the function pointer,
flags, "self" and "parent". Like the current implementation, it has
various METH_ flags for various C signatures. When called, the info from
CCall is matched up (in relatively complex ways) to what the C function
expects.
PEP 590 only adds the "vectorcall". It does away with flags and only has
one C signatures, which is designed to fit all the existing ones, and is
well optimized. Storing the "self"/"parent", and making sure they're
passed to the C function is the responsibility of the callable object.
There's an optimization for "self" (offsetting using
PY_VECTORCALL_ARGUMENTS_OFFSET), and any supporting info can be provided
as part of "self".
I'll reiterate that PEP 590 is more general than PEP 580 and that once
the callable's code has access to the callable object (as both PEPs
allow) then anything is possible. You can't can get more extensible than
that.
Anything is possible, but if one of the possibilities becomes common and
useful, PEP 590 would make it hard to optimize for it.
Python has grown many "METH_*" signatures over the years as we found
more things that need to be passed to callables. Why would
"METH_VECTORCALL" be the last? If it won't (if you think about it as one
more way to call functions), then dedicating a tp_* slot to it sounds
quite expensive.
In one of the ways to call C functions in PEP 580, the function gets
access to:
- the arguments,
- "self", the object
- the class that the method was found in (which is not necessarily
type(self))
I still have to read the details, but when combined with
LOAD_METHOD/CALL_METHOD optimization (avoiding creation of a "bound
method" object), it seems impossible to do this efficiently with just
the callable's code and callable's object.
I would argue the opposite: PEP 590 defines a fixed protocol that is not
easy to extend. PEP 580 on the other hand uses a new data structure
PyCCallDef which could easily be extended in the future (this will
intentionally never be part of the stable ABI, so we can do that).
I have also argued before that the generality of PEP 590 is a bad thing
rather than a good thing: by defining a more rigid protocol as in PEP
580, more optimizations are possible.
PEP 580 has the same limitation for the same reasons. The limitation is
necessary for correctness if an object supports calls via `__call__` and
through another calling convention.
I don't think that this limitation is needed in either PEP. As I
explained at the top of this email, it can easily be solved by not using
the protocol for Python classes. What is wrong with my proposal in PEP
580: https://www.python.org/dev/peps/pep-0580/#inheritance
I'll add Jeroen's notes from the review of the proposed PEP 590
(https://github.com/python/peps/pull/960):
The statement "PEP 580 is specifically targetted at function-like
objects, and doesn't support other callables like classes, partial
functions, or proxies" is factually false. The motivation for PEP 580 is
certainly function/method-like objects but it's a general protocol that
every class can implement. For certain classes, it may not be easy or
desirable to do that but it's always possible.
Given that `PY_METHOD_DESCRIPTOR` is a flag for tp_flags, shouldn't it
be called `Py_TPFLAGS_METHOD_DESCRIPTOR` or something?
Py_TPFLAGS_HAVE_VECTOR_CALL should be Py_TPFLAGS_HAVE_VECTORCALL, to be
consistent with tp_vectorcall_offset and other uses of "vectorcall" (not
"vector call")
And mine, so far:
I'm not clear on the constness of the "args" array.
If it is mutable (PyObject **), you can't, for example, directly pass a
tuple's storage (or any other array that could be used in the call).
If it is not (PyObject * const *), you can't insert the "self" argument in.
The reference implementations seems to be inconsistent here. What's the
intention?
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
