Re: [Python-Dev] PEP 447: add type.__locallookup__
On 13 Sep, 2013, at 12:42, Nick Coghlan ncogh...@gmail.com wrote: Perhaps __getdescriptor__ would work as the method name? Yes, it can technically return a non-descriptor, but the *primary* purpose is to customise the retrieval of objects that will be checked to see if they're descriptors. It *won't* be invoked when looking for ordinary attributes in an instance dict, but *will* be invoked when looking on the class object. __getdescriptor__ would work. The name is not 100% accurate, but a lot clearer than the other alternatives I've seen. Ronald P.S. Sorry about the slow response, its hard to find enough time to seriously participate in the discussion. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 13 Sep, 2013, at 18:19, Steve Dower steve.do...@microsoft.com wrote: From: Steven D'Aprano On Fri, Sep 13, 2013 at 04:26:06AM +, Steve Dower wrote: Last I checked, looking up in the instance dict us exactly what it does. Even the example you posted is doing that. The example from the PEP shows: return cls.__dict__[name] not self.__dict__[name]. It is true that the instance in this case refers to it being an instance of the metaclass, but that instance is, in fact, a class/type. That's why we normally call it cls in a metaclass method rather than self. Right, but where's the difference between these two? class A: def __tdb__(self, name): if name == 'some_attribute_on_my_instance_of_A': return its_value try: return self.__dict__[name] except KeyError: raise AttributeError(name) class MetaB: def __tdb__(cls, name): if name == 'some_attribute_on_my_class_B': return its_value try: return cls.__dict__[name] except KeyError: raise AttributeError(name) (Yes, either of these could be written with __getattribute__, but that function cannot be called by super().) As I see it, there are two (correct) ways to interpret what this method is for, which influences what it should be called. 1. It directly replaces obj.__dict__[name] everywhere that is done, including internally in the interpreter. 2. It is the same as __getattribute__ without the final call to object.__getattribute__ I guess it's also correct to view it as a special helper for super(), but it is more generally applicable than that. It directly replaces cls.__dict__[name] for object.__getattribute__ and super.__getattribute__. The primary reason for writing a proposal is that __getattribute__ can be used to override attribute lookup on an instance, but there is way to override how super() looks up an attribute. Using the method for both super().__getattribute__ and object.__getattribute__ results in a cleaner model than just having a new hook for super().__getattribute__. [...] By the way, I think the PEP should have a more complex example. The SillyObject example is nice and easy to understand, but it doesn't really help with the motivating use-case dynamic classes that can grow new methods on demand. Ronald, if you're reading this, can you add such an example please? Also, there's a typo in the SillyObject M method (fourtytwo should not have a U in it). Agreed. No harm in more examples. [...] A full example of where this may realistically be needed is longer and certainly involves metaclasses, but fundamentally it's just the same as __getattribute__ with slightly different semantics. PyObjC will be a truly realistic example, but that involves loads of fairly complex C code and likely won't help to explain anything beyond (hopefully) showing that this proposal can lead to significant code removal in some situations. Ronald ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 14 Sep, 2013, at 8:30, Nick Coghlan ncogh...@gmail.com wrote: [... interesting text that I'll respond to later ...] So my proposed name is based on the idea that what Ronald is after with the PEP is a hook that *only* gets invoked when the interpreter is doing this hunt for descriptors, but *not* for ordinary attribute lookups. I'll describe the usecase that led to my proposal. I'm amongst other the primary author (and sole maintainer) of PyObjC, which is a bridge between the Python and Objective-C class/object models. PyObjC defines a proxy class for every (Objective-C) class in the Objective-C runtime and when an Objective-C object is made available to Python code the PyObjC bridge creates a proxy object that is an instance of this proxy object. This is simular what wxWidgets or PyQt do, but everything is done at runtime by introspecting the Objective-C runtime. The first time a method is called the bridge looks for an Objective-C selector with the same name and adds that to the class dictionary. This works fine for normal method lookups, by overriding __getattribute__, but causes problems with super: super happily ignores __getattribute__ and peeks in the class __dict__ which may not yet contain the name we're looking for and that can result in incorrect results (both incorrect AttributeErrors and totally incorrect results when the name is not yet present in the parent class' __dict__ but is in the grandparent's __dict__). The current release of PyObjC solves this problem by providing a subclass of super (objc.super) that should be used instead of the builtin one and that works, although the implementation of this class is a gross hack. Users's of PyObjC are currently oblivious of the problem because I sneak in objc.super as a module's globals()['super'] when it imports PyObjC in the currently prefered way (which is using *-imports: from Cocoa import *). I'm currently migrating to deprecating *-imports because those are bad for the usual reasons, but also because framework binding modules are huge and using plain imports makes it possible to delay, and often even avoid, the cost of loading stuff from the framework binding modules. That switch will however make the problem of using __builtin__.super extremely visible for PyObjC users. That, and the implementation hack of objc.super, is why I started looking for a way to cleanly provide a way to hook into the attribute resolution for super(), which ended up as PEP 447. Note that the description is slightly simplified from what PyObjC reall does, Objective-C can (and does) have class and instance methods with the same name, because of that all PyObjC classes have a meta class of the same name and that makes everything even more complicated, but that should not be important for the previous description. PyObjC also enables implementing Objective-C classes in Python, but that's also not relevant for this discussion. Ronald ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 19 September 2013 10:32, Ronald Oussoren ronaldousso...@mac.com wrote: The first time a method is called the bridge looks for an Objective-C selector with the same name and adds that to the class dictionary. This works fine for normal method lookups, by overriding __getattribute__, but causes problems with super: super happily ignores __getattribute__ and peeks in the class __dict__ which may not yet contain the name we're looking for and that can result in incorrect results (both incorrect AttributeErrors and totally incorrect results when the name is not yet present in the parent class' __dict__ but is in the grandparent's __dict__). As an alternative approach, could you use a custom dict subclass as the class __dict__, and catch the peeking in the class __dict__ that way? Or is this one of those places where only a real dict will do? Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 19 Sep, 2013, at 12:00, Paul Moore p.f.mo...@gmail.com wrote: On 19 September 2013 10:32, Ronald Oussoren ronaldousso...@mac.com wrote: The first time a method is called the bridge looks for an Objective-C selector with the same name and adds that to the class dictionary. This works fine for normal method lookups, by overriding __getattribute__, but causes problems with super: super happily ignores __getattribute__ and peeks in the class __dict__ which may not yet contain the name we're looking for and that can result in incorrect results (both incorrect AttributeErrors and totally incorrect results when the name is not yet present in the parent class' __dict__ but is in the grandparent's __dict__). As an alternative approach, could you use a custom dict subclass as the class __dict__, and catch the peeking in the class __dict__ that way? Or is this one of those places where only a real dict will do? The C code uses PyDict_GetItem and AFAIK that doesn't look for a __getitem__ implementation in a subclass. Ronald Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 19 Sep 2013 20:00, Paul Moore p.f.mo...@gmail.com wrote: On 19 September 2013 10:32, Ronald Oussoren ronaldousso...@mac.com wrote: The first time a method is called the bridge looks for an Objective-C selector with the same name and adds that to the class dictionary. This works fine for normal method lookups, by overriding __getattribute__, but causes problems with super: super happily ignores __getattribute__ and peeks in the class __dict__ which may not yet contain the name we're looking for and that can result in incorrect results (both incorrect AttributeErrors and totally incorrect results when the name is not yet present in the parent class' __dict__ but is in the grandparent's __dict__). As an alternative approach, could you use a custom dict subclass as the class __dict__, and catch the peeking in the class __dict__ that way? Or is this one of those places where only a real dict will do? Even Python 3 doesn't let you control the *runtime* type of the class dict, only the type used during evaluation of the class body. I've played with changing that - it makes for a rather special interpreter experience :) Cheers, Nick. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 19 Sep, 2013, at 12:12, Nick Coghlan ncogh...@gmail.com wrote: On 19 Sep 2013 20:00, Paul Moore p.f.mo...@gmail.com wrote: On 19 September 2013 10:32, Ronald Oussoren ronaldousso...@mac.com wrote: The first time a method is called the bridge looks for an Objective-C selector with the same name and adds that to the class dictionary. This works fine for normal method lookups, by overriding __getattribute__, but causes problems with super: super happily ignores __getattribute__ and peeks in the class __dict__ which may not yet contain the name we're looking for and that can result in incorrect results (both incorrect AttributeErrors and totally incorrect results when the name is not yet present in the parent class' __dict__ but is in the grandparent's __dict__). As an alternative approach, could you use a custom dict subclass as the class __dict__, and catch the peeking in the class __dict__ that way? Or is this one of those places where only a real dict will do? Even Python 3 doesn't let you control the *runtime* type of the class dict, only the type used during evaluation of the class body. Changing the class dict type from C is easy enough, but as you wrote below doing this gives you an interesting experience. Changing PyDict_* to call the subclass implementation of the corresponding slot would be easy enough, but that changes the behavior of a core CPython API and I wouldn't look forward to auditing the CPython code base to check if such a change is safe (let alone all other extensions). Some code will use the PyDict_* API instead of the abstract API because the former is faster, but at least some callers for PyDict_GetItem will do this to explicitly get the base class implementation. Ronald I've played with changing that - it makes for a rather special interpreter experience :) Cheers, Nick. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 14 Sep, 2013, at 8:30, Nick Coghlan ncogh...@gmail.com wrote: but the *primary* purpose is to customise the retrieval of objects that will be checked to see if they're descriptors. If that's the case, the PEP should make that clear. Technically, that's what Currently object.__getattribute__ and super.__getattribute__ peek in the __dict__ of classes on the MRO for a class when looking for an attribute. means. However, I agree the current wording only conveys that to the handful of people that already know exactly when in the attribute lookup sequence that step occurs, which is a rather niche audience :) I've been fooling around with this long enough that I forgot that not everyone knows this :-). I guess I'd better include a clearer and more complete description of the current attribute resolution protocol and how my proposal affects that. A nice readable Python implementation of that protocol would be nice to have regardless of the fate of this PEP. Ronald ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On Thu, Sep 19, 2013 at 4:12 AM, Nick Coghlan ncogh...@gmail.com wrote: On 19 Sep 2013 20:00, Paul Moore p.f.mo...@gmail.com wrote: On 19 September 2013 10:32, Ronald Oussoren ronaldousso...@mac.com wrote: The first time a method is called the bridge looks for an Objective-C selector with the same name and adds that to the class dictionary. This works fine for normal method lookups, by overriding __getattribute__, but causes problems with super: super happily ignores __getattribute__ and peeks in the class __dict__ which may not yet contain the name we're looking for and that can result in incorrect results (both incorrect AttributeErrors and totally incorrect results when the name is not yet present in the parent class' __dict__ but is in the grandparent's __dict__). As an alternative approach, could you use a custom dict subclass as the class __dict__, and catch the peeking in the class __dict__ that way? Or is this one of those places where only a real dict will do? Even Python 3 doesn't let you control the *runtime* type of the class dict, only the type used during evaluation of the class body. I've played with changing that - it makes for a rather special interpreter experience :) Same here. :) The PyDict_* API is not your friend for that. It's why I gave up on using a C OrderedDict for tp_dict (opting for a __definition_order__ attribute on classes instead). -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On Thu, Sep 19, 2013 at 4:04 AM, Ronald Oussoren ronaldousso...@mac.comwrote: The C code uses PyDict_GetItem and AFAIK that doesn't look for a __getitem__ implementation in a subclass. Yeah, the PyDict_* API is definitely not subclass friendly. :( -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On Thu, Sep 19, 2013 at 3:49 AM, Ronald Oussoren ronaldousso...@mac.comwrote: On 14 Sep, 2013, at 8:30, Nick Coghlan ncogh...@gmail.com wrote: However, I agree the current wording only conveys that to the handful of people that already know exactly when in the attribute lookup sequence that step occurs, which is a rather niche audience :) I've been fooling around with this long enough that I forgot that not everyone knows this :-). I guess I'd better include a clearer and more complete description of the current attribute resolution protocol and how my proposal affects that. A nice readable Python implementation of that protocol would be nice to have regardless of the fate of this PEP. +1 -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 13 September 2013 22:23, Steven D'Aprano st...@pearwood.info wrote:
On Fri, Sep 13, 2013 at 08:42:46PM +1000, Nick Coghlan wrote:
Perhaps __getdescriptor__ would work as the method name? Yes, it can
technically return a non-descriptor,
So technically that name is, um, what's the term... oh yes, a lie.
:-)
In this case, __getdescriptor__ means I am looking for a
descriptor, please don't invoke the descriptor methods or traverse the
MRO, just return the raw object, not you *must* give me a
descriptor.
The name suggestion comes from the fact that name bindings on the
instance and names bindings on the class are *different*, in that only
the latter participate in the descriptor protocol:
class A:
... def m(self): pass
...
A.m
function A.m at 0x7fb51ad63320
a = A()
a.f = A.m
a.m
bound method A.m of __main__.A object at 0x7fb51ad60550
a.f
function A.m at 0x7fb51ad63320
It's the same function object underneath, so what's going on?
The trick is that only *types* get to play the descriptor game, where
the interpreter looks for __get__, __set__ and __delete__ on the
returned object and invokes them with found. Ordinary instances don't
have that behaviour - instead, they go through type(self) to look for
descriptors, and anything they find in the instance dictionary is
returned unaltered.
This difference in how attribute lookups are handled is actually the
most fundamental difference between normal class instances and classes
themselves (which are instances of metaclasses).
but the *primary* purpose is to
customise the retrieval of objects that will be checked to see if they're
descriptors.
If that's the case, the PEP should make that clear.
Technically, that's what Currently object.__getattribute__ and
super.__getattribute__ peek in the __dict__ of classes on the MRO for
a class when looking for an attribute. means.
However, I agree the current wording only conveys that to the handful
of people that already know exactly when in the attribute lookup
sequence that step occurs, which is a rather niche audience :)
It's also why I like __getdescriptor__ as a name - it's based on *why*
we're doing the lookup, rather than *how* we expect it to be done.
[Aside: the PEP states that the method shouldn't invoke descriptors.
What's the reason for that? If I take the statement literally, doesn't
it mean that the method mustn't use any other methods at all? Surely
that can't be what is intended, but I'm not sure what is intended.]
It means it shouldn't invoke __get__, __set__ or __delete__ on the
returned object, since that's the responsibility of the caller.
For example, there are times when a descriptor will be retrieved to
check for the presence of a __set__ or __delete__ method, but never
actually have its methods invoked because it is shadowed in the
instance dictionary:
class Shadowable:
... def __get__(self, *args):
... print(Shadowable.__get__ called!)
...
class Enforced:
... def __get__(self, *args):
... print(Enforced.__get__ called!)
... def __set__(self, *args):
... print(Enforced.__set__ called!)
...
class Example:
... s = Shadowable()
... e = Enforced()
... def __getattribute__(self, attr):
... print(Retrieving {} from class.format(attr))
... return super().__getattribute__(attr)
...
x = Example()
x.s
Retrieving s from class
Shadowable.__get__ called!
x.s = 1
x.s
Retrieving s from class
1
This is the key line: we retrieved 's' from the class, but *didn't*
invoke the __get__ method because it was shadowed in the instance
dictionary.
It works this way because *if* the descriptor defines __set__ or
__delete__, then Python will *ignore* the instance variable:
x.e
Retrieving e from class
Enforced.__get__ called!
x.__dict__[e] = 1
Retrieving __dict__ from class
x.e
Retrieving e from class
Enforced.__get__ called!
So my proposed name is based on the idea that what Ronald is after
with the PEP is a hook that *only* gets invoked when the interpreter
is doing this hunt for descriptors, but *not* for ordinary attribute
lookups.
It *won't* be invoked when looking for ordinary attributes in
an instance dict, but *will* be invoked when looking on the class object.
Just to be clear, if I have:
instance = MyClass()
x = instance.name
and name is found in instance.__dict__, then this special method will
not be invoked. But if name is not found in the instance dict, then
name will be looked up on the class object MyClass, which may invoke
this special method. Am I correct?
Well, that's *my* proposal. While re-reading the current PEP, I
realised my suggested change actually goes quite a bit further than
just proposing a different name: unlike the current PEP, my advice is
that the new hook should NOT be invoked for instance attribute lookups
and should *not* replace looking directly into the class dict.
Instead, it would be the descriptor lookup counterpart to __getattr__:
whereas __getattr__ only
Re: [Python-Dev] PEP 447: add type.__locallookup__
On Fri, Sep 13, 2013 at 04:26:06AM +, Steve Dower wrote: Last I checked, looking up in the instance dict us exactly what it does. Even the example you posted is doing that. The example from the PEP shows: return cls.__dict__[name] not self.__dict__[name]. It is true that the instance in this case refers to it being an instance of the metaclass, but that instance is, in fact, a class/type. That's why we normally call it cls in a metaclass method rather than self. I was reacting to your statement that [quote]Putting type or class in the name would be misleading[end quote]. I don't believe it is misleading. type lookup is exactly what it does: it does a lookup on a type. Take your example below: Consider class A, which knows it has a method F, but will not create it until the first __getattribute__ call. Now class B derives from A, and someone calls super(B).F(obj). [Aside: I'm not sure why you are using an unbound super object super(B) instead of the more usual super(B, obj).F(). Have I missed something?] As I understand it, that ends up calling type(B).__whatever__(B, 'F'). So the objects being used are: - the metaclass type(B); - the class B but not the instance self = B(), even though F is a regular instance method on A that ends up seeing self as the first argument. Given this, I believe that lookup on the type is exactly what the method does, whether you interpret the type as the metaclass (the __whatever__ method is called on the metaclass) or the class B (which ends up as the first argument to the whatever method). By the way, I think the PEP should have a more complex example. The SillyObject example is nice and easy to understand, but it doesn't really help with the motivating use-case dynamic classes that can grow new methods on demand. Ronald, if you're reading this, can you add such an example please? Also, there's a typo in the SillyObject M method (fourtytwo should not have a U in it). [...] There may still be valid questions to answer (such as, should overrides if this method on base classes be inherited), but whether it is a type/class method is no longer one of those. I don't believe that anyone is arguing that it should be a class method. I'm certainly not. But even if somebody is, that doesn't have anything to do with the name. We write dict.fromkeys(), not dict.typefromkeys(). -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
Perhaps __getdescriptor__ would work as the method name? Yes, it can technically return a non-descriptor, but the *primary* purpose is to customise the retrieval of objects that will be checked to see if they're descriptors. It *won't* be invoked when looking for ordinary attributes in an instance dict, but *will* be invoked when looking on the class object. Cheers, Nick. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On Fri, Sep 13, 2013 at 08:42:46PM +1000, Nick Coghlan wrote: Perhaps __getdescriptor__ would work as the method name? Yes, it can technically return a non-descriptor, So technically that name is, um, what's the term... oh yes, a lie. :-) but the *primary* purpose is to customise the retrieval of objects that will be checked to see if they're descriptors. If that's the case, the PEP should make that clear. [Aside: the PEP states that the method shouldn't invoke descriptors. What's the reason for that? If I take the statement literally, doesn't it mean that the method mustn't use any other methods at all? Surely that can't be what is intended, but I'm not sure what is intended.] It *won't* be invoked when looking for ordinary attributes in an instance dict, but *will* be invoked when looking on the class object. Just to be clear, if I have: instance = MyClass() x = instance.name and name is found in instance.__dict__, then this special method will not be invoked. But if name is not found in the instance dict, then name will be looked up on the class object MyClass, which may invoke this special method. Am I correct? -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
From: Steven D'Aprano On Fri, Sep 13, 2013 at 04:26:06AM +, Steve Dower wrote: Last I checked, looking up in the instance dict us exactly what it does. Even the example you posted is doing that. The example from the PEP shows: return cls.__dict__[name] not self.__dict__[name]. It is true that the instance in this case refers to it being an instance of the metaclass, but that instance is, in fact, a class/type. That's why we normally call it cls in a metaclass method rather than self. Right, but where's the difference between these two? class A: def __tdb__(self, name): if name == 'some_attribute_on_my_instance_of_A': return its_value try: return self.__dict__[name] except KeyError: raise AttributeError(name) class MetaB: def __tdb__(cls, name): if name == 'some_attribute_on_my_class_B': return its_value try: return cls.__dict__[name] except KeyError: raise AttributeError(name) (Yes, either of these could be written with __getattribute__, but that function cannot be called by super().) As I see it, there are two (correct) ways to interpret what this method is for, which influences what it should be called. 1. It directly replaces obj.__dict__[name] everywhere that is done, including internally in the interpreter. 2. It is the same as __getattribute__ without the final call to object.__getattribute__ I guess it's also correct to view it as a special helper for super(), but it is more generally applicable than that. [...] By the way, I think the PEP should have a more complex example. The SillyObject example is nice and easy to understand, but it doesn't really help with the motivating use-case dynamic classes that can grow new methods on demand. Ronald, if you're reading this, can you add such an example please? Also, there's a typo in the SillyObject M method (fourtytwo should not have a U in it). Agreed. No harm in more examples. Here's a quick example of code that does not behave correctly at present. class A: def __getattribute__(self, name): if name == 'foo': return 'A.foo' return object.__getattribute__(self, name) class B(A): def get_foo(self): return super().foo B().get_foo() # skips A.__getattribute__ Traceback (most recent call last): File stdin, line 1, in module File stdin, line 3, in get_foo AttributeError: 'super' object has no attribute 'foo' B().foo A.foo After changing to use the __tbd__ method: class A: def __getattribute__(self, name): '''Not strictly necessary for this example, but I would expect that most types overriding __tbd__ also want to override __getattribute__ to use it. Or maybe object.__getattribute__ should be changed to use __tbd__ too...?''' try: return self.__tbd__(name) except AttributeError: return object.__getattribute__(self, name) def __tbd__(self, name):# CHANGED if name == 'foo': return 'A.foo' try: return self.__dict__[name] except KeyError: raise AttributeError(name) # CHANGED class B(A): def get_foo(self): return super().foo B().get_foo() # does not skip A.__tbd__ A.foo # hopefully this is the result :) B().foo A.foo A full example of where this may realistically be needed is longer and certainly involves metaclasses, but fundamentally it's just the same as __getattribute__ with slightly different semantics. Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 9 Sep, 2013, at 20:23, Jan Kaliszewski z...@chopin.edu.pl wrote: Is '__locallookup__' a really good name? In Python, *local* -- especially in context of *lookups* -- usually associates with locals() i.e. a namespace of a function/method execution frame or a namespace of a class, during *definition* of that class... So '__locallookup__' can be confusing. Why not just '__getclassattribute__' or '__classlookup__', or '__classattribute__'...? I don't particularly like __locallookup__ either, but haven't found a better name yet. __lookup_in_class__ was the best alternative I could come up with, and that feels different than other special methods. The name in the PEP is more or less derived from _PyType_Lookup, with local meaning only in this class, don't recurse in the rest of the MRO. Ronald ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 9 Sep, 2013, at 17:43, Mark Shannon m...@hotpy.org wrote: I would like time to investigate this further, but at the moment I think it will either make attribute lookup poorly defined or slow. Of the top of my head, the problem as a I see it is basically this: Currently, type.__getattribute__() is a fixed point in the lookup of attributes. The proposal means that a fixed point is not reached until the cls parameter of type.__getattribute__() is either object or type, otherwise type.__getattribute__() and type.__locallookup__ must bounce back and forth. This will slow down *every* attribute lookup for what is a fairly obscure use case. I did a benchmark run (see the pep for details) and that seems to indicate that the performance impact is very small, possibly because the patch keeps the attribute lookup cache used by _PyType_Lookup. Anyway, I'm glad that there is now some real discussion on the proposal. Not unsurprisingly I'd love to have this, or something simular, in 3.4. I had hoped to repost the PEP a while back with more information on how the API would affect PyObjC (code removal, performance impact), but haven't had time to move forward on that front :-( Ronald Cheers, Mark. On 09/09/13 16:27, Guido van Rossum wrote: Let's just accept this PEP. It looks like a nice addition to the metaclass machinery and I don't think we'll get much more useful feedback by waiting. On Mon, Sep 9, 2013 at 7:30 AM, Ethan Furman et...@stoneleaf.us mailto:et...@stoneleaf.us wrote: On 07/30/2013 11:17 PM, Ronald Oussoren wrote: And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? *Bump*. It would be nice if this could make into 3.4. -- ~Ethan~ _ Python-Dev mailing list Python-Dev@python.org mailto:Python-Dev@python.org https://mail.python.org/__mailman/listinfo/python-dev https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/__mailman/options/python-dev/__guido%40python.org https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido http://python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mark%40hotpy.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On Thu, 12 Sep 2013 16:42:39 +0200, Ronald Oussoren ronaldousso...@mac.com wrote: On 9 Sep, 2013, at 20:23, Jan Kaliszewski z...@chopin.edu.pl wrote: Is '__locallookup__' a really good name? In Python, *local* -- especially in context of *lookups* -- usually associates with locals() i.e. a namespace of a function/method execution frame or a namespace of a class, during *definition* of that class... So '__locallookup__' can be confusing. Why not just '__getclassattribute__' or '__classlookup__', or '__classattribute__'...? I don't particularly like __locallookup__ either, but haven't found a better name yet. __lookup_in_class__ was the best alternative I could come up with, and that feels different than other special methods. The name in the PEP is more or less derived from _PyType_Lookup, with local meaning only in this class, don't recurse in the rest of the MRO. Why is __getclassattribute__ worse than __locallookup__? --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 9 Sep, 2013, at 23:18, Mark Shannon m...@hotpy.org wrote: On 09/09/13 15:30, Ethan Furman wrote: On 07/30/2013 11:17 PM, Ronald Oussoren wrote: And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? *Bump*. It would be nice if this could make into 3.4. IMO, there are some issues that need to be addressed before PEP 447 should be accepted. 1. Is there even a problem at all, or is this just a bug in super? Why doesn't super() respect the __getattribute__ method of the superclass? Because __getattribute__ looks in the instance __dict__ before walking the MRO, while super does not. 2. Is this the best way to solve the problem (if there is a problem)? Would a __super__ special method be sufficient and less intrusive. One reason for the __locallookup__ method is to make normal and super attribute lookup more simular, adding a __super__ special method would lead to code duplication: both __getattribute__ and __super__ would either contain simular code, or would call out to a shared method anyway. 3. Are the proposed semantics OK? I think they are, but very low level changes such as these can have unforeseen consequences. For example, PEP 3135 and issue 12370. 4. What is the performance impact. pybench really doesn't count as a benchmark. What kind of benchmark would you like to see? BTW. I ran more than pybench, I also ran the part of the performance benchmark that worked on py3k at the time. Ronald ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On Thu, Sep 12, 2013 at 04:42:39PM +0200, Ronald Oussoren wrote: I don't particularly like __locallookup__ either, but haven't found a better name yet. __lookup_in_class__ was the best alternative I could come up with, and that feels different than other special methods. The name in the PEP is more or less derived from _PyType_Lookup, with local meaning only in this class, don't recurse in the rest of the MRO. How about __typelookup__ ? Surely that's the obvious name to derive from _PyType_Lookup :-) -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 12 sep. 2013, at 17:30, R. David Murray rdmur...@bitdance.com wrote: On Thu, 12 Sep 2013 16:42:39 +0200, Ronald Oussoren ronaldousso...@mac.com wrote: On 9 Sep, 2013, at 20:23, Jan Kaliszewski z...@chopin.edu.pl wrote: Is '__locallookup__' a really good name? In Python, *local* -- especially in context of *lookups* -- usually associates with locals() i.e. a namespace of a function/method execution frame or a namespace of a class, during *definition* of that class... So '__locallookup__' can be confusing. Why not just '__getclassattribute__' or '__classlookup__', or '__classattribute__'...? I don't particularly like __locallookup__ either, but haven't found a better name yet. __lookup_in_class__ was the best alternative I could come up with, and that feels different than other special methods. The name in the PEP is more or less derived from _PyType_Lookup, with local meaning only in this class, don't recurse in the rest of the MRO. Why is __getclassattribute__ worse than __locallookup__? Getclassattribute feels like it is related to classmethod, or fetches an attribute of a class. The method does however fetch a value from the class that is transformed to the actual attribute value through the descriptor protocol. Ronald --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
What about __getlocalattribute__ or __getattributenorecurse__? Long, but this isn't going to be used often. Putting type or class in the name would be misleading. It's an instance method (that is most useful when implemented on a metaclass). (Apologies for the top post.) Sent from my Windows Phone From: Steven D'Apranomailto:st...@pearwood.info Sent: 9/12/2013 16:09 To: python-dev@python.orgmailto:python-dev@python.org Subject: Re: [Python-Dev] PEP 447: add type.__locallookup__ On Thu, Sep 12, 2013 at 04:42:39PM +0200, Ronald Oussoren wrote: I don't particularly like __locallookup__ either, but haven't found a better name yet. __lookup_in_class__ was the best alternative I could come up with, and that feels different than other special methods. The name in the PEP is more or less derived from _PyType_Lookup, with local meaning only in this class, don't recurse in the rest of the MRO. How about __typelookup__ ? Surely that's the obvious name to derive from _PyType_Lookup :-) -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On Fri, Sep 13, 2013 at 03:04:49AM +, Steve Dower wrote: What about __getlocalattribute__ or __getattributenorecurse__? Long, but this isn't going to be used often. This has nothing to do with locals, nor does it have anything to do with recursion, so both those names are misleading. Putting type or class in the name would be misleading. It's an instance method (that is most useful when implemented on a metaclass). Regardless of whether it is an instance method or not, by default it performs the lookup on the type. Hence the C function _PyType_Lookup and hence my suggestion __typelookup__. But I think that __typelookup__ does describe quite well what the method does. It looks up on the type. The PEP is fairly clear on how this is supposed to work, e.g. the default type.__whatever__ method will look up in the class/type dict. PEP 447 includes an example of how you might implement this in Python: class MetaType(type): def __locallookup__(cls, name): try: return cls.__dict__[name] except KeyError: raise AttributeError(name) from None local lookup doesn't even come close to describing what the method does or why you would use it. It suggests something to do with locals, which is not the case. Neither does __getattributenorecurse__, which suggests looking up an attribute on an object without following the inheritance hierarchy, e.g. looking in the instance __dict__ but not the class __dict__. So the complete opposite of what it actually does. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
Last I checked, looking up in the instance dict us exactly what it does. Even the example you posted is doing that. And the only difference from __getattribute__ is that it throws instead of following the MRO, which is intended to allow base classes (via super, and another call to this method) to dynamically respond to a getattr without the cooperation of subclasses. Consider class A, which knows it has a method F, but will not create it until the first __getattribute__ call. Now class B derives from A, and someone calls super(B).F(obj). Currently, super only looks in __dict__ for F, which will fail to invoke A.__getattribute__. Because super is used to provide MRO traversal, it can't rely on B.__getattribute__ to perform the traversal, so it currently has no choice. The PEP was originally adding a special class method to provide a __getattribute__ equivalent that would not traverse the MRO, so that super could use it and people can create dynamic classes that can act as base classes. I pointed out that this could be an instance method (thereby avoid automatic-classmethod magic) and implemented on a metaclass for the class behavior. Unless the PEP has changed recently, this is still an instance method that will look up members defined directly on the type and not on base classes. There may still be valid questions to answer (such as, should overrides if this method on base classes be inherited), but whether it is a type/class method is no longer one of those. Cheers, Steve Sent from my Windows Phone From: Steven D'Apranomailto:st...@pearwood.info Sent: 9/12/2013 21:00 To: python-dev@python.orgmailto:python-dev@python.org Subject: Re: [Python-Dev] PEP 447: add type.__locallookup__ On Fri, Sep 13, 2013 at 03:04:49AM +, Steve Dower wrote: What about __getlocalattribute__ or __getattributenorecurse__? Long, but this isn't going to be used often. This has nothing to do with locals, nor does it have anything to do with recursion, so both those names are misleading. Putting type or class in the name would be misleading. It's an instance method (that is most useful when implemented on a metaclass). Regardless of whether it is an instance method or not, by default it performs the lookup on the type. Hence the C function _PyType_Lookup and hence my suggestion __typelookup__. But I think that __typelookup__ does describe quite well what the method does. It looks up on the type. The PEP is fairly clear on how this is supposed to work, e.g. the default type.__whatever__ method will look up in the class/type dict. PEP 447 includes an example of how you might implement this in Python: class MetaType(type): def __locallookup__(cls, name): try: return cls.__dict__[name] except KeyError: raise AttributeError(name) from None local lookup doesn't even come close to describing what the method does or why you would use it. It suggests something to do with locals, which is not the case. Neither does __getattributenorecurse__, which suggests looking up an attribute on an object without following the inheritance hierarchy, e.g. looking in the instance __dict__ but not the class __dict__. So the complete opposite of what it actually does. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
Hi Mark, On Mon, Sep 9, 2013 at 11:18 PM, Mark Shannon m...@hotpy.org wrote: 5. Other implementations. What do the Jython/IronPython/PyPy developers think? Thanks for asking :-) I'm fine with staying out of language design issues like this one, and I believe it's the general concensus in PyPy. Whatever gets decided will probably be easy to port to PyPy and have no measurable performance impact: the extra checks on the type, if any, are all constant-folded by the JIT. A bientôt, Armin. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 07/30/2013 11:17 PM, Ronald Oussoren wrote: And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? *Bump*. It would be nice if this could make into 3.4. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
Let's just accept this PEP. It looks like a nice addition to the metaclass machinery and I don't think we'll get much more useful feedback by waiting. On Mon, Sep 9, 2013 at 7:30 AM, Ethan Furman et...@stoneleaf.us wrote: On 07/30/2013 11:17 PM, Ronald Oussoren wrote: And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? *Bump*. It would be nice if this could make into 3.4. -- ~Ethan~ __**_ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/**mailman/listinfo/python-devhttps://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/**mailman/options/python-dev/** guido%40python.orghttps://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
OK, how much time do you need? --Guido van Rossum (sent from Android phone) On Sep 9, 2013 8:44 AM, Mark Shannon m...@hotpy.org wrote: I would like time to investigate this further, but at the moment I think it will either make attribute lookup poorly defined or slow. Of the top of my head, the problem as a I see it is basically this: Currently, type.__getattribute__() is a fixed point in the lookup of attributes. The proposal means that a fixed point is not reached until the cls parameter of type.__getattribute__() is either object or type, otherwise type.__getattribute__() and type.__locallookup__ must bounce back and forth. This will slow down *every* attribute lookup for what is a fairly obscure use case. Cheers, Mark. On 09/09/13 16:27, Guido van Rossum wrote: Let's just accept this PEP. It looks like a nice addition to the metaclass machinery and I don't think we'll get much more useful feedback by waiting. On Mon, Sep 9, 2013 at 7:30 AM, Ethan Furman et...@stoneleaf.us mailto: et...@stoneleaf.us wrote: On 07/30/2013 11:17 PM, Ronald Oussoren wrote: And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? *Bump*. It would be nice if this could make into 3.4. -- ~Ethan~ __**___ Python-Dev mailing list Python-Dev@python.org mailto:Python-Dev@python.org https://mail.python.org/__**mailman/listinfo/python-devhttps://mail.python.org/__mailman/listinfo/python-dev https://mail.python.org/**mailman/listinfo/python-devhttps://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/__** mailman/options/python-dev/__**guido%40python.orghttps://mail.python.org/__mailman/options/python-dev/__guido%40python.org https://mail.python.org/**mailman/options/python-dev/**guido%40python.orghttps://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido http://python.org/~guido) __**_ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/**mailman/listinfo/python-devhttps://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/**mailman/options/python-dev/** mark%40hotpy.orghttps://mail.python.org/mailman/options/python-dev/mark%40hotpy.org __**_ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/**mailman/listinfo/python-devhttps://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/**mailman/options/python-dev/** guido%40python.orghttps://mail.python.org/mailman/options/python-dev/guido%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
I'll look into it this evening. On 09/09/13 17:03, Guido van Rossum wrote: OK, how much time do you need? --Guido van Rossum (sent from Android phone) On Sep 9, 2013 8:44 AM, Mark Shannon m...@hotpy.org mailto:m...@hotpy.org wrote: I would like time to investigate this further, but at the moment I think it will either make attribute lookup poorly defined or slow. Of the top of my head, the problem as a I see it is basically this: Currently, type.__getattribute__() is a fixed point in the lookup of attributes. The proposal means that a fixed point is not reached until the cls parameter of type.__getattribute__() is either object or type, otherwise type.__getattribute__() and type.__locallookup__ must bounce back and forth. This will slow down *every* attribute lookup for what is a fairly obscure use case. Cheers, Mark. On 09/09/13 16:27, Guido van Rossum wrote: Let's just accept this PEP. It looks like a nice addition to the metaclass machinery and I don't think we'll get much more useful feedback by waiting. On Mon, Sep 9, 2013 at 7:30 AM, Ethan Furman et...@stoneleaf.us mailto:et...@stoneleaf.us mailto:et...@stoneleaf.us mailto:et...@stoneleaf.us wrote: On 07/30/2013 11:17 PM, Ronald Oussoren wrote: And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? *Bump*. It would be nice if this could make into 3.4. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org mailto:Python-Dev@python.org mailto:Python-Dev@python.org mailto:Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev https://mail.python.org/__mailman/listinfo/python-dev https://mail.python.org/__mailman/listinfo/python-dev https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org https://mail.python.org/__mailman/options/python-dev/__guido%40python.org https://mail.python.org/__mailman/options/python-dev/__guido%40python.org https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido http://python.org/~guido http://python.org/~guido) _ Python-Dev mailing list Python-Dev@python.org mailto:Python-Dev@python.org https://mail.python.org/__mailman/listinfo/python-dev https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/__mailman/options/python-dev/__mark%40hotpy.org https://mail.python.org/mailman/options/python-dev/mark%40hotpy.org _ Python-Dev mailing list Python-Dev@python.org mailto:Python-Dev@python.org https://mail.python.org/__mailman/listinfo/python-dev https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/__mailman/options/python-dev/__guido%40python.org https://mail.python.org/mailman/options/python-dev/guido%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
I would like time to investigate this further, but at the moment I think it will either make attribute lookup poorly defined or slow. Of the top of my head, the problem as a I see it is basically this: Currently, type.__getattribute__() is a fixed point in the lookup of attributes. The proposal means that a fixed point is not reached until the cls parameter of type.__getattribute__() is either object or type, otherwise type.__getattribute__() and type.__locallookup__ must bounce back and forth. This will slow down *every* attribute lookup for what is a fairly obscure use case. Cheers, Mark. On 09/09/13 16:27, Guido van Rossum wrote: Let's just accept this PEP. It looks like a nice addition to the metaclass machinery and I don't think we'll get much more useful feedback by waiting. On Mon, Sep 9, 2013 at 7:30 AM, Ethan Furman et...@stoneleaf.us mailto:et...@stoneleaf.us wrote: On 07/30/2013 11:17 PM, Ronald Oussoren wrote: And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? *Bump*. It would be nice if this could make into 3.4. -- ~Ethan~ _ Python-Dev mailing list Python-Dev@python.org mailto:Python-Dev@python.org https://mail.python.org/__mailman/listinfo/python-dev https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/__mailman/options/python-dev/__guido%40python.org https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido http://python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/mark%40hotpy.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 09/09/2013 08:43 AM, Mark Shannon wrote: I would like time to investigate this further, but at the moment I think it will either make attribute lookup poorly defined or slow. Of the top of my head, the problem as a I see it is basically this: Currently, type.__getattribute__() is a fixed point in the lookup of attributes. The proposal means that a fixed point is not reached until the cls parameter of type.__getattribute__() is either object or type, otherwise type.__getattribute__() and type.__locallookup__ must bounce back and forth. This will slow down *every* attribute lookup for what is a fairly obscure use case. Looks like there's a patch we can try at http://bugs.python.org/issue18181. Here are Ronald's last timings: --- PYBENCH 2.1 --- * using CPython 3.4.0a0 (default, Jul 29 2013, 13:01:34) [GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] * disabled garbage collection * system check interval set to maximum: 2147483647 * using timer: time.perf_counter * timer: resolution=1e-09, implementation=clock_gettime(CLOCK_MONOTONIC) --- Benchmark: pep447.pybench --- Rounds: 10 Warp: 10 Timer: time.perf_counter Machine Details: Platform ID: Linux-2.6.32-358.114.1.openstack.el6.x86_64-x86_64-with-centos-6.4-Final Processor: x86_64 Python: Implementation: CPython Executable: /tmp/default-pep447/bin/python3 Version:3.4.0a0 Compiler: GCC 4.4.7 20120313 (Red Hat 4.4.7-3) Bits: 64bit Build: Jul 29 2013 14:09:12 (#default) Unicode:UCS4 --- Comparing with: default.pybench --- Rounds: 10 Warp: 10 Timer: time.perf_counter Machine Details: Platform ID: Linux-2.6.32-358.114.1.openstack.el6.x86_64-x86_64-with-centos-6.4-Final Processor: x86_64 Python: Implementation: CPython Executable: /tmp/default/bin/python3 Version:3.4.0a0 Compiler: GCC 4.4.7 20120313 (Red Hat 4.4.7-3) Bits: 64bit Build: Jul 29 2013 13:01:34 (#default) Unicode:UCS4 Test minimum run-timeaverage run-time thisother diffthisother diff --- BuiltinFunctionCalls:45ms44ms +1.3%45ms44ms +1.3% BuiltinMethodLookup:26ms27ms -2.4%27ms27ms -2.2% CompareFloats:33ms34ms -0.7%33ms34ms -1.1% CompareFloatsIntegers:66ms67ms -0.9%66ms67ms -0.8% CompareIntegers:51ms50ms +0.9%51ms50ms +0.8% CompareInternedStrings:34ms33ms +0.4%34ms34ms -0.4% CompareLongs:29ms29ms -0.1%29ms29ms -0.0% CompareStrings:43ms44ms -1.8%44ms44ms -1.8% ComplexPythonFunctionCalls:44ms42ms +3.9%44ms42ms +4.1% ConcatStrings:33ms33ms -0.4%33ms33ms -1.0% CreateInstances:47ms48ms -2.9%47ms49ms -3.4% CreateNewInstances:35ms36ms -2.5%36ms36ms -2.5% CreateStringsWithConcat:69ms70ms -0.7%69ms70ms -0.9% DictCreation:52ms50ms +3.1%52ms50ms +3.0% DictWithFloatKeys:40ms44ms -10.1%43ms45ms -5.8% DictWithIntegerKeys:32ms36ms -11.2%35ms37ms -4.6% DictWithStringKeys:29ms34ms -15.7%35ms40ms -11.0% ForLoops:30ms29ms +2.2%30ms29ms +2.2% IfThenElse:38ms41ms -6.7%38ms41ms -6.9% ListSlicing:36ms36ms -0.7%36ms37ms -1.3%
Re: [Python-Dev] PEP 447: add type.__locallookup__
On Mon, 09 Sep 2013 09:05:57 -0700 Ethan Furman et...@stoneleaf.us wrote: On 09/09/2013 08:43 AM, Mark Shannon wrote: I would like time to investigate this further, but at the moment I think it will either make attribute lookup poorly defined or slow. Of the top of my head, the problem as a I see it is basically this: Currently, type.__getattribute__() is a fixed point in the lookup of attributes. The proposal means that a fixed point is not reached until the cls parameter of type.__getattribute__() is either object or type, otherwise type.__getattribute__() and type.__locallookup__ must bounce back and forth. This will slow down *every* attribute lookup for what is a fairly obscure use case. Looks like there's a patch we can try at http://bugs.python.org/issue18181. Here are Ronald's last timings: Thanks but can you run a benchmark that actually exercises the feature? (I don't know enough about it to know what that would be, but I suppose it has to do with lookups on classes, rather than on instances) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
Is '__locallookup__' a really good name? In Python, *local* -- especially in context of *lookups* -- usually associates with locals() i.e. a namespace of a function/method execution frame or a namespace of a class, during *definition* of that class... So '__locallookup__' can be confusing. Why not just '__getclassattribute__' or '__classlookup__', or '__classattribute__'...? Cheers. *j ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
Yes, I don't like the 'local' prefix too. How about '__dictlookup__'? It's just more self-describing. Yury On 2013-09-09, at 2:23 PM, Jan Kaliszewski z...@chopin.edu.pl wrote: Is '__locallookup__' a really good name? In Python, *local* -- especially in context of *lookups* -- usually associates with locals() i.e. a namespace of a function/method execution frame or a namespace of a class, during *definition* of that class... So '__locallookup__' can be confusing. Why not just '__getclassattribute__' or '__classlookup__', or '__classattribute__'...? Cheers. *j ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
Since the main problem is super(), maybe we can just add a __super__ method to get a custom super implementation? 2013/9/9 Ethan Furman et...@stoneleaf.us: On 07/30/2013 11:17 PM, Ronald Oussoren wrote: And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? *Bump*. It would be nice if this could make into 3.4. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/benjamin%40python.org -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 09/09/13 15:30, Ethan Furman wrote: On 07/30/2013 11:17 PM, Ronald Oussoren wrote: And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? *Bump*. It would be nice if this could make into 3.4. IMO, there are some issues that need to be addressed before PEP 447 should be accepted. 1. Is there even a problem at all, or is this just a bug in super? Why doesn't super() respect the __getattribute__ method of the superclass? 2. Is this the best way to solve the problem (if there is a problem)? Would a __super__ special method be sufficient and less intrusive. 3. Are the proposed semantics OK? I think they are, but very low level changes such as these can have unforeseen consequences. For example, PEP 3135 and issue 12370. 4. What is the performance impact. pybench really doesn't count as a benchmark. 5. Other implementations. What do the Jython/IronPython/PyPy developers think? Cheers, Mark. p.s. Apologies for top-posting earlier ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
2013/9/9 Mark Shannon m...@hotpy.org: On 09/09/13 15:30, Ethan Furman wrote: On 07/30/2013 11:17 PM, Ronald Oussoren wrote: And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? *Bump*. It would be nice if this could make into 3.4. IMO, there are some issues that need to be addressed before PEP 447 should be accepted. 1. Is there even a problem at all, or is this just a bug in super? Why doesn't super() respect the __getattribute__ method of the superclass? You want to be looking things up on the class, not an instance. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 09/09/13 22:25, Benjamin Peterson wrote:
2013/9/9 Mark Shannon m...@hotpy.org:
On 09/09/13 15:30, Ethan Furman wrote:
On 07/30/2013 11:17 PM, Ronald Oussoren wrote:
And something I forgot to ask: is anyone willing to be the
BDFL-Delegate for
PEP 447?
*Bump*.
It would be nice if this could make into 3.4.
IMO, there are some issues that need to be addressed before PEP 447 should
be accepted.
1. Is there even a problem at all, or is this just a bug in super?
Why doesn't super() respect the __getattribute__ method of the superclass?
You want to be looking things up on the class, not an instance.
Sorry, I meant 'type of the superclass' rather than 'superclass'.
I was suggesting that super().m should be
type(type(self).__mro__[1]).__getattribute__('m')
rather than type(self).__mro__[1].__dict__['m']
(ignoring descriptor __get__ calls)
Unfortunately this brings its own problems, due to __getattribute__
doing its own traversal of the mro.
So, scratch point 1.
Cheers,
Mark.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 29 Jul, 2013, at 14:49, Ronald Oussoren ronaldousso...@mac.com wrote: Hi, This PEP proposed to add a __locallookup__ slot to type objects, which is used by _PyType_Lookup and super_getattro instead of peeking in the tp_dict of classes. The PEP text explains why this is needed. Differences with the previous version: * Better explanation of why this is a useful addition * type.__locallookup__ is no longer optional. * I've added benchmarking results using pybench. (using the patch attached to issue 18181) Ronald And something I forgot to ask: is anyone willing to be the BDFL-Delegate for PEP 447? Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 447: add type.__locallookup__
Hi, This PEP proposed to add a __locallookup__ slot to type objects, which is used by _PyType_Lookup and super_getattro instead of peeking in the tp_dict of classes. The PEP text explains why this is needed. Differences with the previous version: * Better explanation of why this is a useful addition * type.__locallookup__ is no longer optional. * I've added benchmarking results using pybench. (using the patch attached to issue 18181) Ronald PEP: 447 Title: Add __locallookup__ method to metaclass Version: $Revision$ Last-Modified: $Date$ Author: Ronald Oussoren ronaldousso...@mac.com Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 12-Jun-2013 Post-History: 2-Jul-2013, 15-Jul-2013, 29-Jul-2013 Abstract Currently ``object.__getattribute__`` and ``super.__getattribute__`` peek in the ``__dict__`` of classes on the MRO for a class when looking for an attribute. This PEP adds an optional ``__locallookup__`` method to a metaclass that can be used to override this behavior. Rationale = It is currently not possible to influence how the `super class`_ looks up attributes (that is, ``super.__getattribute__`` unconditionally peeks in the class ``__dict__``), and that can be problematic for dynamic classes that can grow new methods on demand. The ``__locallookup__`` method makes it possible to dynamicly add attributes even when looking them up using the `super class`_. The new method affects ``object.__getattribute__`` (and `PyObject_GenericGetAttr`_) as well for consistency. Background -- The current behavior of ``super.__getattribute__`` causes problems for classes that are dynamic proxies for other (non-Python) classes or types, an example of which is `PyObjC`_. PyObjC creates a Python class for every class in the Objective-C runtime, and looks up methods in the Objective-C runtime when they are used. This works fine for normal access, but doesn't work for access with ``super`` objects. Because of this PyObjC currently includes a custom ``super`` that must be used with its classes. The API in this PEP makes it possible to remove the custom ``super`` and simplifies the implementation because the custom lookup behavior can be added in a central location. The superclass attribute lookup hook Both ``super.__getattribute__`` and ``object.__getattribute__`` (or `PyObject_GenericGetAttr`_ in C code) walk an object's MRO and peek in the class' ``__dict__`` to look up attributes. A way to affect this lookup is using a method on the meta class for the type, that by default looks up the name in the class ``__dict__``. In Python code -- A meta type can define a method ``__locallookup__`` that is called during attribute resolution by both ``super.__getattribute__`` and ``object.__getattribute``:: class MetaType(type): def __locallookup__(cls, name): try: return cls.__dict__[name] except KeyError: raise AttributeError(name) from None The ``__locallookup__`` method has as its arguments a class and the name of the attribute that is looked up. It should return the value of the attribute without invoking descriptors, or raise `AttributeError`_ when the name cannot be found. The `type`_ class provides a default implementation for ``__locallookup__``, that looks up the name in the class dictionary. Example usage . The code below implements a silly metaclass that redirects attribute lookup to uppercase versions of names:: class UpperCaseAccess (type): def __locallookup__(cls, name): return cls.__dict__[name.upper()] class SillyObject (metaclass=UpperCaseAccess): def m(self): return 42 def M(self): return fourtytwo obj = SillyObject() assert obj.m() == fortytwo In C code - A new slot ``tp_locallookup`` is added to the ``PyTypeObject`` struct, this slot corresponds to the ``__locallookup__`` method on `type`_. The slot has the following prototype:: PyObject* (*locallookupfunc)(PyTypeObject* cls, PyObject* name); This method should lookup *name* in the namespace of *cls*, without looking at superclasses, and should not invoke descriptors. The method returns ``NULL`` without setting an exception when the *name* cannot be found, and returns a new reference otherwise (not a borrowed reference). Use of this hook by the interpreter --- The new method is required for metatypes and as such is defined on `type_`. Both ``super.__getattribute__`` and ``object.__getattribute__``/`PyObject_GenericGetAttr`_ (through ``_PyType_Lookup``) use the this ``__locallookup__`` method when walking the MRO. Other changes to the implementation --- The change for `PyObject_GenericGetAttr`_ will be done by changing the private function ``_PyType_Lookup``. This currently returns a
Re: [Python-Dev] PEP 447: add type.__locallookup__
Hi, Le Mon, 29 Jul 2013 14:49:18 +0200, Ronald Oussoren ronaldousso...@mac.com a écrit : Hi, This PEP proposed to add a __locallookup__ slot to type objects, which is used by _PyType_Lookup and super_getattro instead of peeking in the tp_dict of classes. The PEP text explains why this is needed. Differences with the previous version: * Better explanation of why this is a useful addition * type.__locallookup__ is no longer optional. * I've added benchmarking results using pybench. (using the patch attached to issue 18181) Could you please run the whole benchmark suite? http://hg.python.org/benchmarks/ Thanks Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 29 Jul, 2013, at 14:58, Antoine Pitrou solip...@pitrou.net wrote: Hi, Le Mon, 29 Jul 2013 14:49:18 +0200, Ronald Oussoren ronaldousso...@mac.com a écrit : Hi, This PEP proposed to add a __locallookup__ slot to type objects, which is used by _PyType_Lookup and super_getattro instead of peeking in the tp_dict of classes. The PEP text explains why this is needed. Differences with the previous version: * Better explanation of why this is a useful addition * type.__locallookup__ is no longer optional. * I've added benchmarking results using pybench. (using the patch attached to issue 18181) Could you please run the whole benchmark suite? http://hg.python.org/benchmarks/ Sure. Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 29 Jul, 2013, at 15:07, Ronald Oussoren ronaldousso...@mac.com wrote: On 29 Jul, 2013, at 14:58, Antoine Pitrou solip...@pitrou.net wrote: Hi, Le Mon, 29 Jul 2013 14:49:18 +0200, Ronald Oussoren ronaldousso...@mac.com a écrit : Hi, This PEP proposed to add a __locallookup__ slot to type objects, which is used by _PyType_Lookup and super_getattro instead of peeking in the tp_dict of classes. The PEP text explains why this is needed. Differences with the previous version: * Better explanation of why this is a useful addition * type.__locallookup__ is no longer optional. * I've added benchmarking results using pybench. (using the patch attached to issue 18181) Could you please run the whole benchmark suite? http://hg.python.org/benchmarks/ Sure. That's harder than I had expected, when I use the make_perf3.sh to create a python 3 compatible version of the benchmark suite and then run the suite it craps out because it cannnot find spitfire, which isn't translated (as are several other benchmarks). I'll have to investigate why the suite doesn't work. Ronald Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
Le Mon, 29 Jul 2013 15:26:41 +0200, Ronald Oussoren ronaldousso...@mac.com a écrit : On 29 Jul, 2013, at 15:07, Ronald Oussoren ronaldousso...@mac.com wrote: On 29 Jul, 2013, at 14:58, Antoine Pitrou solip...@pitrou.net wrote: Hi, Le Mon, 29 Jul 2013 14:49:18 +0200, Ronald Oussoren ronaldousso...@mac.com a écrit : Hi, This PEP proposed to add a __locallookup__ slot to type objects, which is used by _PyType_Lookup and super_getattro instead of peeking in the tp_dict of classes. The PEP text explains why this is needed. Differences with the previous version: * Better explanation of why this is a useful addition * type.__locallookup__ is no longer optional. * I've added benchmarking results using pybench. (using the patch attached to issue 18181) Could you please run the whole benchmark suite? http://hg.python.org/benchmarks/ Sure. That's harder than I had expected, when I use the make_perf3.sh to create a python 3 compatible version of the benchmark suite and then run the suite it craps out because it cannnot find spitfire, which isn't translated (as are several other benchmarks). Ah, sorry. I think it's enough to run the 2n3 subsuite here, though. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
