[Python-Dev] PEP 447: hooking into super() attribute resolution
Hi,
I've posted a new update of my proposal to add a way to override the attribute
resolution proces by super(). I've rewritten the PEP and implementation based
on feedback by Steve Dower.
In the current edition of the proposal the hook is a method on the type,
defined in a metaclass for types. A simple demo:
class meta (type):
def __locallookup__(cls, name):
return type.__locallookup__(cls, name.upper())
class S (metaclass=meta):
def m(self):
return 42
def M(self):
return fortytwo
class SS (S):
def m(self):
return 21
def M(self):
return twentyone
o = SS()
print(self, o.m())
print(super, super(SS, o).m())
The last line currently prints super 42 and would print super fortytwo with
my proposal.
A major open issue: the __locallookup__ method could also be used for normal
attribute resolution, but that probably causes issues with attribute caching
(see the PEP for details). I haven't worked out yet if it is worthwhile to
tweak the proposal to fix the caching issues (even though the proposal
currently says that PyObject_GenericGetAttr will use the new method). Fixing
the caching issue definitly would help in my primary use case by reducing code
duplication, but might end up making the API unnecessarily complex.
Ronald
PEP: 447
Title: Hooking into super attribute resolution
Version: $Revision$
Last-Modified: $Date$
Author: Ronald Oussoren ronaldousso...@mac.com
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Jun-2013
Post-History: 2-Jul-2013, ?
Abstract
In current python releases the attribute resolution of the `super class`_
peeks in the ``__dict__`` attribute of classes on the MRO to look
for attributes. This PEP introduces a hook that classes can use
to override that behavior for specific classes.
Rationale
=
Peeking in the class ``__dict__`` works for regular classes, but can
cause problems when a class dynamicly looks up attributes in a
``__getattribute__`` method.
This new hook makes it possible to affect attribute lookup for both normal
attribute lookup and lookup through the `super class`_.
The superclass attribute lookup hook
Both ``super.__getattribute__`` and ``object.__getattribute__`` (or
`PyObject_GenericGetAttr`_ in C code) walk an object's MRO and peek in the
class' ``__dict__`` to look up attributes. A way to affect this lookup is
using a method on the meta class for the type, that by default looks up
the name in the class ``__dict__``.
In Python code
--
A meta type can define a method ``__locallookup__`` that is called during
attribute resolution by both ``super.__getattribute__`` and
``object.__getattribute``::
class MetaType (type):
def __locallookup__(cls, name):
try:
return cls.__dict__[name]
except KeyError:
raise AttributeError(name) from None
The example method above is pseudo code for the implementation of this method on
`type`_ (the actual implementation is in C, and doesn't suffer from the
recursion
problem in this example).
The ``__locallookup__`` method has as its arguments a class and the name of the
attribute
that is looked up. It should return the value of the attribute without invoking
descriptors,
or raise `AttributeError`_ when the name cannot be found.
In C code
-
A new slot ``tp_locallookup`` is added to the ``PyTypeObject`` struct, this slot
corresponds to the ``__locallookup__`` method on `type`_.
The slot has the following prototype::
PyObject* (*locallookupfunc)(PyTypeObject* cls, PyObject* name);
This method should lookup *name* in the namespace of *cls*, without looking at
superclasses,
and should not invoke descriptors. The method returns ``NULL`` without setting
an exception
when the *name* cannot be found, and returns a new reference otherwise (not a
borrowed reference).
Usage of this hook
--
The new method will be defined for `type`_, and will peek in the class
dictionary::
static PyObject*
type_locallookup(PyTypeObject* cls, PyObject* name)
{
PyObject* res;
if (!cls-tp_dict) {
return NULL;
}
res = PyDict_GetItem(cls, name);
Py_XINCREF(res);
return res;
}
The new method will be used by both `PyObject_GenericGetAttr`_ and
``super.__getattribute__`` instead of peeking in a type's ``tp_dict``.
Alternative proposals
-
``__getattribute_super__``
..
An earlier version of this PEP used the following static method on classes::
def __getattribute_super__(cls, name, object, owner): pass
This method performed name lookup as well as invoking descriptors and was
necessarily
limited to working only with ``super.__getattribute__``.
Reuse ``tp_getattro``
.
It
Re: [Python-Dev] PEP 447: hooking into super() attribute resolution
On Mon, Jul 15, 2013 at 8:12 AM, Ronald Oussoren ronaldousso...@mac.com wrote: I've posted a new update of my proposal to add a way to override the attribute resolution proces by super(). I've rewritten the PEP and implementation based on feedback by Steve Dower. In the current edition of the proposal the hook is a method on the type, defined in a metaclass for types. A simple demo: class meta (type): def __locallookup__(cls, name): return type.__locallookup__(cls, name.upper()) class S (metaclass=meta): def m(self): return 42 def M(self): return fortytwo class SS (S): def m(self): return 21 def M(self): return twentyone o = SS() print(self, o.m()) print(super, super(SS, o).m()) The last line currently prints super 42 and would print super fortytwo with my proposal. I would suggest adding that motivation to the PEP, since you clearly thought it was necessary to get people to consider your proposal. :-) A major open issue: the __locallookup__ method could also be used for normal attribute resolution, but that probably causes issues with attribute caching (see the PEP for details). I haven't worked out yet if it is worthwhile to tweak the proposal to fix the caching issues (even though the proposal currently says that PyObject_GenericGetAttr will use the new method). Fixing the caching issue definitly would help in my primary use case by reducing code duplication, but might end up making the API unnecessarily complex. Hm. It looks like the functionality you actually want to hook into is in _PyType_Lookup(). I think that in this case the PEP's acceptance will be highly correlated with your ability to produce an actual patch that (a) implements exactly the functionality you want (never mind whether it matches the exact API currently proposed), and (b) doesn't slow down classes that don't provide this hook. Other than that, I think that it's a laudable attempt at generalizing, and I hope you solve the implementation conundrum. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: hooking into super() attribute resolution
On 15 Jul, 2013, at 18:49, Guido van Rossum gu...@python.org wrote: A major open issue: the __locallookup__ method could also be used for normal attribute resolution, but that probably causes issues with attribute caching (see the PEP for details). I haven't worked out yet if it is worthwhile to tweak the proposal to fix the caching issues (even though the proposal currently says that PyObject_GenericGetAttr will use the new method). Fixing the caching issue definitly would help in my primary use case by reducing code duplication, but might end up making the API unnecessarily complex. Hm. It looks like the functionality you actually want to hook into is in _PyType_Lookup(). That's right. I didn't want to get too technical, but forgot to consider who are reading this :-) I think that in this case the PEP's acceptance will be highly correlated with your ability to produce an actual patch that (a) implements exactly the functionality you want (never mind whether it matches the exact API currently proposed), and (b) doesn't slow down classes that don't provide this hook. I'd only reall need the functional change to super(), but I am working on a patch that also changes _PyType_Lookup. I think I can avoid slowdown by making the tp_locallookup optional and only punishing those classes that use the new slot. A minor complication is that I'll have to change the interface of _PyType_Lookup, it currently returns a borrowed reference and will return a new reference. That's just careful bookkeeping though. Other than that, I think that it's a laudable attempt at generalizing, and I hope you solve the implementation conundrum. I was pleasantly surprised in how the changed API was cleaner and applicable to _PyType_Lookup as well. I guess that means I'm on the right path. Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: hooking into super() attribute resolution
On Mon, Jul 15, 2013 at 9:56 AM, Ronald Oussoren ronaldousso...@mac.com wrote: On 15 Jul, 2013, at 18:49, Guido van Rossum gu...@python.org wrote: A major open issue: the __locallookup__ method could also be used for normal attribute resolution, but that probably causes issues with attribute caching (see the PEP for details). I haven't worked out yet if it is worthwhile to tweak the proposal to fix the caching issues (even though the proposal currently says that PyObject_GenericGetAttr will use the new method). Fixing the caching issue definitly would help in my primary use case by reducing code duplication, but might end up making the API unnecessarily complex. Hm. It looks like the functionality you actually want to hook into is in _PyType_Lookup(). That's right. I didn't want to get too technical, but forgot to consider who are reading this :-) Heh. :-) I think that in this case the PEP's acceptance will be highly correlated with your ability to produce an actual patch that (a) implements exactly the functionality you want (never mind whether it matches the exact API currently proposed), and (b) doesn't slow down classes that don't provide this hook. I'd only reall need the functional change to super(), but I am working on a patch that also changes _PyType_Lookup. I think I can avoid slowdown by making the tp_locallookup optional and only punishing those classes that use the new slot. A minor complication is that I'll have to change the interface of _PyType_Lookup, it currently returns a borrowed reference and will return a new reference. That's just careful bookkeeping though. Hm. That sounds scary. I'd rename it, because this is actually an important (and scary) signature change, that type checkers otherwise won't see. Don't trust that you can find all call sites (there may be some in 3rd party code even though it starts with _). You don't want to suddenly start causing leaks all over the place. FWIW I agree that the change should affect _PyType_Lookup, because it would be very surprising if super() used a different override than regular attribute lookup. Other than that, I think that it's a laudable attempt at generalizing, and I hope you solve the implementation conundrum. I was pleasantly surprised in how the changed API was cleaner and applicable to _PyType_Lookup as well. I guess that means I'm on the right path. There's a Zen of Python line about that. :-) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
