Guido van Rossum wrote:
> On Wed, Apr 2, 2008 at 6:30 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote:
>> One of the issues with porting to Py3k is the problem that __getattr__
>> and __getattribute__ can't reliably provide special methods like __add__
>> the way __getattr__ could with classic classes. (As first noted by Terry
>> Reedy years ago, and recently seeing some new activity on the bug
>> tracker [1])
>>
>> The culprit here is the fact that __getattribute__ and its associated
>> machinery is typically never invoked for the methods with dedicated tp_*
>> slots in the C-level type structure.
>
> Well, yes, this is all an intentional part of the new-style class design.
Not complaining, just trying to provide some background for those that
may not be quite as familiar with the inner workings of typeobject.c :)
>> What do people think of the idea of providing an extra method on type
>> objects that goes through all of the C-level special method slots, and
>> for each one that isn't currently set, does a getattr() on the
>> associated special name and stores the result (if any) on the current
>> type object?
>
> Does a getattr on what? Since you seem to be thinking specifically of
> proxies here, I'm thinking you're doing a getattr on an *instance* --
> but it seems wrong to base the *type* slots on that.
D'oh, you're right - the specific proxying example I am thinking of (see
below) does indeed grab bound methods directly from the underlying
instance. However, I think the idea is salvageable (whether or not it is
*worth* salvaging is of course a completely different question!).
>> When converting a proxy class that relies on __getattr__ from classic
>
> Can you show specific code for such a proxy class? I'm having a hard
> time imagining how it would work (not having used proxies in a really
> long time...).
From tempfile._TemporaryFileWrapper, which aims to delegate as many
operations as it can automatically to the underlying file object:
def __getattr__(self, name):
# Attribute lookups are delegated to the underlying file
# and cached for non-numeric results
# (i.e. methods are cached, closed and friends are not)
file = self.__dict__['file']
a = getattr(file, name)
if not issubclass(type(a), type(0)):
setattr(self, name, a)
return a
For 2.x, the only methods that need to be overridden explicitly are
those where this bound method caching does the wrong thing (__exit__ and
__enter__ needed to be on that list, which is what first brought this
class to my attention). For 3.0, it was also necessary to add:
def __iter__(self):
return iter(self.file)
It wasn't too bad in this case since file doesn't implement many tp_*
slots, but the 3.0 version of classes that delegate a lot of operations
to a specific member variable will be a lot more verbose in any cases
where the underlying type being delegated to implements some of the
number or container protocols.
>> to new-style, all that would then be needed is to invoke the new method on
>> the class object after defining the class (a class decorator or
>> metaclass could be provided somewhere to make this a bit tidier).
>
> Hm. So you are thinking of a proxy for a class?!?!
Sort of - I'm thinking mainly of classes like _TemporaryFileWrapper that
delegate most operations to a specific member variable, and expect that
member variable to always be of a specific type.
> Note that if you set a class attribute corresponding to a special
> method (e.g. C.__add__ = ...) the corresponding C slot is
> automatically updated, so you should be able to write a class
> decorator or mixin or helper function to do this in pure Python,
> unless I completely misunderstand what you're after.
Yeah, doing it in typeobject was mostly an easy way of getting at the
complete list of special methods with tp_* slots without having to
maintain two copies of that list.
>> This seems a lot cleaner than expecting everyone that implements a proxy
>> object to maintain there own list of all of the relevant special
>> methods, and locates the implementation support in an area of the code
>> that already has plenty of infrastructure dedicated to keeping Python
>> visible attributes in sync with the C visible tp_* slots.
>
> How many proxy implementations does the world need? Maybe we should
> add one to the stdlib?
I don't know enough about the different ways people proxy or otherwise
delegate special methods to know if it is feasible to provide a
one-size-fits-most implementation in the standard library.
That said, maybe it would be enough if a type instance could be queried
for the list of special method names it implements that the interpreter
can access without going through __getattribute__?
Then the slots of a class delegating to a specific type could be
initialised appropriately by doing something like:
for name in delegate_type.special_methods():
if not hasattr(cls, name):
def delegation(*args, **kwds):
self, *args = args # +1 on arbitrary tuple unpacking ;)
getattr(self.delegate, name)(*args, **kwds)
setattr(cls, name, delegation)
The approach I suggested in my original email would instead look more
like this:
class Foo: ...
Foo.delegate_special_methods('delegate', delegate_type)
where delegate_special_methods is basically just a C level
implementation of the loop described above (except that the 'delegation'
callable could be a lot more efficient than the given Python function).
Another option would be to provide an explicit list in the documentation
of the slot names for the tp_* methods which the interpreter may access
without going through __getattr__ and __getattribute__.
The discussion in the bug report that got me thinking about this topic
commented on the fact that quite a few magic methods were added during
the 2.x development cycle - I think the key point I missed at the time
is the fact that most of those *didn't* have corresponding tp_* slots,
so __getattr__ and __getattribute__ (particularly the latter) can
intercept them just fine.
That said, the documentation approach would probably be too limiting on
alternate interpreters though - why should other implementations be
restricted from providing optimised access to special methods just
because we haven't done so certain cases in CPython?
If we don't make any changes at all, the delegation loop shown above can
actually already be written as follows:
for name in dir(delegate_type):
if (name.startswith('__')
and name.endswith('__')
and not hasattr(cls, name)):
def delegation(*args, **kwds):
self, *args = args # +1 on arbitrary tuple unpacking ;)
getattr(self.delegate, name)(*args, **kwds)
setattr(cls, name, delegation)
>> Thoughts? Altenative ideas? Howls of protest?
>
> No, so far just a bit of confusion. :-)
Hopefully the above makes my concerms a bit clearer.
I'm actually hoping to hear from some more people that would benefit
from having better support for this kind of delegation - my interest in
the matter is fairly academic (based solely on the tempfile bugs arising
from the initial conversion to Py3k), so my personal inclination is
actually to put a stronger note in the documentation about the fact that
the lookup of special methods may bypass __getattribute__ entirely and
leave it at that.
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---------------------------------------------------------------
http://www.boredomandlaziness.org
_______________________________________________
Python-3000 mailing list
Python-3000@python.org
http://mail.python.org/mailman/listinfo/python-3000
Unsubscribe:
http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
