Re: AttributeError in with statement (3.2.2)
On 12/16/2011 8:26 PM, Steven D'Aprano wrote: On Fri, 16 Dec 2011 17:05:57 -0500, Terry Reedy wrote: It is am important distinction [unbound versus bound] It is not an important distinction, and I am not confusing the two. So we agree on the distinction but disagree on its importance. Let us leave it at that. Bound or unbound, it is still an instance method. OK. So 'instance method' is a bit ambiguous (more than I thought, or would prefer) in that it can refer to unbound methods, bounds methods, or both. So be it. and that a function defined inside a class is different from a function outside of a class. That, and your repetition of the same claim further on, is a insulting lie. If you can't assume I'm carrying on this discussion in good faith, If you can't assume that I am intelligent and experienced enough to know the meaning of def, one on the most basic aspects of Python, and you are unwilling to give me the benefit of any doubt you might have on that score, but instead go on to 'correct' me publicly, then no, I can't. Def statements always create functions. I have known that for 14 years since the first day I started with Python. I have never thought differently. If you actually think that I have, you are wrong. I'm glad to hear it. But nevertheless you have made statements (which I quoted, and you deleted from your reply) that suggest the opposite. OK, let us look at the my statement and your 'repetition of the same claim further on' that I previously deleted. I wrote These are bound methods. The instance methods are the functions wrapped. As I indicated in response to Ethan, I would now revised the second sentence now to The unbound methods are the function wrapped or The instance-requiring methods are the functions wrapped. But that is not important here. In my opinion, there is no way that anyone reading that in good faith could conclude that I do not know the meaning of def statements. They are not the subject of discussion in that sentence or the rest of this thread. But in response you wrote. I am afraid you are mistaken. About what? You go on to explain. What you say may very well apply to other languages, but in Python, def creates functions no matter where you execute it. Always and without exception. So that makes twice that you said or implied that I think the location of a def statement changes what it creates, even though I explicitly said the opposite when I suggested that the glossary entry might be revised. What am I to think at such a tactic. You are normally much more careful in what you write. If I have misinterpreted them, or if you had worded them badly, there's no need to attribute malice to me. I did not do that. I gave my opinion of your statement, just as you have given your opinions of mine. I really did not and do not know why you misrepresented my knowledge of Python. I actually consider overt intentional malice much less likely than other possibilities. Calling me a liar I did not do that, any more than you have been calling me things. I believe you are asking for the same 'benefit of the doubt' that I believe you denied to me. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Thu, 15 Dec 2011 19:39:17 -0500, Terry Reedy wrote: [...] After reading your post, I think I have worked out where our disagreement lines: you think that bound methods and instance methods are not the same thing, and that a function defined inside a class is different from a function outside of a class. For example, you say: If so, the output is a *bound method*. In your example above, func is not an instance method and obj is not a bound method. It is simply an partially evaluated curried function or if you prefer, a bound function. Take you pick, or make up your own term, but it is NOT an instance method, and later on: So what are methods? In Python, methods are wrappers around functions which automatically pass the instance to the inner function object. These are bound methods. The instance methods are the functions wrapped. I am afraid you are mistaken. What you say may very well apply to other languages, but in Python, def creates functions no matter where you execute it. Always and without exception. I admit to an earlier mistake: I thought that conversion from function to method occurred once, when the class statement was executed, but I was mistaken. Re-reading Raymond Hettinger's excellent article on the descriptor protocol reminded me that methods are created on an as-needed basis, at runtime. Back to methods and def. So let's see what happens in Python 3.1: def spam(self): # Clearly a function. ...pass ... class K(object): ... def ham(self): # Allegedly an instance method ... pass ... According to your various statements, spam is a function and ham is an instance method. Merely storing a outside function inside a class doesn't create an instance method, it creates what you call a bound method (although I'm unclear what you think the difference is). So let's see what happens when we compare an alleged bound method that is not an instance method with an actual instance method: K.spam = spam type(K.spam) is type(K.ham) True Their types are identical. We get the same thing when we compare an actual function with a function object created inside a class, which you claim is actually an instance method: type(spam) is type(K.__dict__['ham']) # Bypass descriptor protocol True Nor is there any difference in type between bound and unbound methods: they are both instance methods differing only in whether or not they have the first argument self available. Bound is an adjective, not part of the type: small method, obfuscated method, buggy method, or bound method. Python 3 no longer uses unbound methods, since they are functionally identical to the unwrapped function, so this snippet is from Python 2.6: py k = K() py type(k.ham) is type(K().ham) # bound vs bound True py type(k.ham) is type(K.ham) # bound vs unbound True In Python, instance methods are wrappers around function objects; they are created on call, and generally do not exist *anywhere* until needed, or if you store a reference to them. k = K() a = k.ham b = k.ham a is b False Under normal circumstances, the only persistent object is the function, which you can extract from the (class or instance) __dict__ or the method wrapper: a.__func__ is K.__dict__['ham'] True Methods are created by the descriptor protocol: when you use the normal a.b syntax to access K.__dict__['ham'], the metaclass calls the __get__ method of function ham. If you call it from an instance, you get a method object bound to that instance. If you call it from the class, in Python 2 you get a method object not bound to an instance, in Python 3 you get the function without a wrapper. [Aside: class methods and static methods also work the same way, via __get__ and the descriptor protocol, but behave differently. Class methods reuse the instance method type, which is somewhat confusing, and static methods just return the function.] So, what the hell does all this mean in practice? Most of the time, absolutely nothing. That's why I started this discussion with the disclaimer that I didn't think it was worth changing the (not quite right) definition of method you originally quoted. Most of the time, we only access methods via the instance.method syntax, which gives us an instance method. By the principle that if it quacks like a duck and swims like a duck, it is good enough to call it a duck, I'm happy to agree that ham here is a method: class K: def ham(self): pass That covers 99% of all use-cases and is good enough for most situations. But the OP ran into one of those edge cases in the 1%, where things are not so simple. He was attempting to create an instance method called __exit__ and store it in the instance __dict__ instead of the class __dict__. This is a perfectly reasonable thing to do, but there are two gotchas to it: * The descriptor protocol doesn't get used for instance lookups. That's why you can't stick properties in an instance,
Re: AttributeError in with statement (3.2.2)
On 12/16/2011 4:22 AM, Steven D'Aprano wrote: On Thu, 15 Dec 2011 19:39:17 -0500, Terry Reedy wrote: [...] After reading your post, I think I have worked out where our disagreement lines: you think that bound methods and instance methods are not the same thing, Do you agree that an unbound method and a bound method are different? In Python, as indicated by the glossary entry, an unspecified 'method' is usually meant to be an unbound method. It is am important distinction and I do not see why you insist on confusing the two. and that a function defined inside a class is different from a function outside of a class. That, and your repetition of the same claim further on, is a insulting lie. Def statements always create functions. I have known that for 14 years since the first day I started with Python. I have never thought differently. If you actually think that I have, you are wrong. What I have said from my first response is that a function that is an attribute of a class, *whether defined in or outside the class*, gets special treatment when accessed via an instance of the class. And that is the truth. If you are ever interested in learning anything from me on this subject, re=read what I already wrote with a more open mind than you have so far. Otherwise, I am done. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
Terry Reedy wrote: On 12/16/2011 4:22 AM, Steven D'Aprano wrote: On Thu, 15 Dec 2011 19:39:17 -0500, Terry Reedy wrote: [...] After reading your post, I think I have worked out where our disagreement lies: you think that bound methods and instance methods are not the same thing, Do you agree that an unbound method and a bound method are different? In Python, as indicated by the glossary entry, an unspecified 'method' is usually meant to be an unbound method. I think you two are in violent agreement as far as how Python is functioning, and the conflict is in the names given to the various pieces... I think a glossary would help (please correct me): function: callable code suite method: function that lives in a class unbound method: function that lives in a class bound method: callable wrapper around function that has been extracted from class that will supply the instance object to the function (note: Python does not save these, they are recreated at each lookup) and here is where I think you two diverge: instance method (Steven): a bound method that has been saved into the instance __dict__ (no matter how created) instance method (Terry): a function that must be looked up in the class Have I missed anything? Honestly-trying-learn-the-distinctions-ly yours, ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
Ethan Furman wrote: Terry Reedy wrote: On 12/16/2011 4:22 AM, Steven D'Aprano wrote: On Thu, 15 Dec 2011 19:39:17 -0500, Terry Reedy wrote: [...] After reading your post, I think I have worked out where our disagreement lies: you think that bound methods and instance methods are not the same thing, Do you agree that an unbound method and a bound method are different? In Python, as indicated by the glossary entry, an unspecified 'method' is usually meant to be an unbound method. I think you two are in violent agreement as far as how Python is functioning, and the conflict is in the names given to the various pieces... I think a glossary would help (please correct me): function: callable code suite method: function that lives in a class unbound method: function that lives in a class bound method: callable wrapper around function that has been extracted from class that will supply the instance object to the function (note: Python does not save these, they are recreated at each lookup) I think the above 'bound method' definition should be attributed to Terry, and Steven's follows: bound method: callable wrapper around any function that will accept an instance object as the first parameter, and the wrapper will supply said instance object when calling the function (and where/how function was created is irrelevent, as is where the wrapper is stored) and here is where I think you two diverge: instance method (Steven): a bound method that has been saved into the instance __dict__ (no matter how created) instance method (Terry): a function that must be looked up in the class Have I missed anything? Honestly-trying-learn-the-distinctions-ly yours, ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Fri, 16 Dec 2011 17:05:57 -0500, Terry Reedy wrote: On 12/16/2011 4:22 AM, Steven D'Aprano wrote: On Thu, 15 Dec 2011 19:39:17 -0500, Terry Reedy wrote: [...] After reading your post, I think I have worked out where our disagreement lines: you think that bound methods and instance methods are not the same thing, Do you agree that an unbound method and a bound method are different? Are different or are different types? Bound and unbound methods in Python 2 are the same type. They are different in the sense that one is effectively a curry of the other, but they are implemented as a single type. So presumably the difference is a bound method has a slot filled and an unbound method doesn't, or some other implementation detail. In Python 3, unbound methods don't exist. In Python, as indicated by the glossary entry, an unspecified 'method' is usually meant to be an unbound method. I think you are badly confused here. The glossary talks about calling methods as an attribute of an instance, e.g. instance.method(), and the method receiving the instance as the first argument. That's a bound method, not unbound. How do you conclude that this indicates that an unspecified method is unbound? Your conclusion is especially nonsensical because unbound methods don't exist in Python 3 -- you're alleging that the usual meaning of method is something which no longer exists in the language! method A function which is defined inside a class body. If called as an attribute of an instance of that class, the method will get the instance object as its first argument (which is usually called self). See function and nested scope. http://docs.python.org/glossary.html It is am important distinction and I do not see why you insist on confusing the two. It is not an important distinction, and I am not confusing the two. Bound or unbound, it is still an instance method. and that a function defined inside a class is different from a function outside of a class. That, and your repetition of the same claim further on, is a insulting lie. If you can't assume I'm carrying on this discussion in good faith, then we have a serious problem. This is not like you Terry -- I've been reading your posts for many years, and you're not usually so obstreperous. Def statements always create functions. I have known that for 14 years since the first day I started with Python. I have never thought differently. If you actually think that I have, you are wrong. I'm glad to hear it. But nevertheless you have made statements (which I quoted, and you deleted from your reply) that suggest the opposite. If I have misinterpreted them, or if you had worded them badly, there's no need to attribute malice to me. Calling me a liar for something said in good faith and which I prefixed with I think is simply not cool. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Fri, 16 Dec 2011 15:26:30 -0800, Ethan Furman wrote: Terry Reedy wrote: On 12/16/2011 4:22 AM, Steven D'Aprano wrote: [...] I think you two are in violent agreement as far as how Python is functioning, and the conflict is in the names given to the various pieces... I think a glossary would help (please correct me): function: callable code suite Sure, why not? Informally: a function is some object which can be called and returns a result. Hence we sometimes call str() and int() functions even though they are actually implemented as types. Callable would be better for such informal use, but newbies may not recognise the term. To be more precise: a function is a particular type of callable object created by the def and lambda statements. method: function that lives in a class Informally, I'm happy with that definition, and use it myself. Under 99% of circumstances, it would be overly pedantic to describe spam in the following as anything other than a method: class C: def spam(self): pass Call it the duck-type principle: since spam quacks like a duck, we might as well call it a duck. If you only care about the easy cases, then spam is a method and we're done. But as a formal definition, it fails on two fronts: functions that live inside classes are not actually methods, they are functions, and while methods are commonly found in classes, they can also be found in instances, or anywhere else for that matter. Functions inside classes are still functions I said that since spam quacks like a method, we might as well call it a method. But it doesn't swim like a method: it's actually still a function with a remarkable power of mimicry. There are places and times where the fact that spam is not actually a method comes through. The three most important examples in my opinion are: * inside the class suite, during class definition time, if you use or inspect spam, you will see it is a function and not a method; * if you bypass the getattr mechanism for looking up attributes, e.g. use C.__dict__['spam'] instead of C().spam, you will see that spam is stored in the dict as a function object, not a method object; * (Python 3 only) if use the standard getattr mechanism, but call it on the class instead of an instance, you will get the function object instead of a method. So, to be precise: spam is actually a function. It is stored as a function inside C.__dict__, but when you retrieve it using C().spam Python automagically builds a method object wrapping spam and gives you that instead. This distinction is important, well, hardly ever, but on the rare occasion it is important, you need to know it. One example of when it may be important: because methods are created as needed, you can't assume that identity checks will necessarily pass for methods: `inst.spam is inst.spam` may not be true. Methods can be created outside of classes - The normal way to get a method is by defining it inside a class. In Java, I believe that is the *only* way to get methods, but this is Python, and we have other options. Methods are first class objects, which means they have a type and a constructor, so if you can get hold of their type, you can make them by hand. You can retrieve the method type two ways: * call type on a method you made the usual way: type(C().spam); * or import types and use types.MethodType. Confusingly, the method type calls itself instancemethod but is exported via the types module as MethodType. Oh well. MethodType requires two arguments, In Python 2, it accepts an optional third argument. Since Python 2 is the past, I'll only talk about the Python 3 version, where the signature might look something like this: MethodType(callable, instance) = instancemethod object This creates a wrapper around callable (usually, but not necessarily a function) and binds instance to that wrapper so the first argument (conventionally called self) can be automatically provided. Another way to create methods is by calling the function object __get__ method by hand. That's what gets used in the normal C().spam case. So you can create methods without putting them inside a class. One trick is that you can customise behaviour on a per-instance basis by overriding the normal method living inside the class with one which lives inside the instance: class C: def spam(self): return Spam spam spam, lovely SPAM!!! c = C() c.spam = types.MethodType(lambda self: Bloody vikings!, c) Instance c now has a custom method which overrides C.spam. [Aside #1: I think this was the original point of contention between Terry and I. For reasons I still don't understand, it seems to me that he denies that the per-instance method spam is actually a method, since it doesn't live inside the class.] [Aside #2: this per-instance method trick may not work for
Re: AttributeError in with statement (3.2.2)
On Dec 14, 9:01 pm, Steven D'Aprano steve +comp.lang.pyt...@pearwood.info wrote: [...] So what are methods? In Python, methods are wrappers around functions which automatically pass the instance to the inner function object. Under normal circumstances, you create methods by declaring functions inside a class, but that's not the only way to create methods, and it is not the only place they can be found. I've always understood methods as basically function wrappers that pass in the instance, so it's good to hear somebody else formulate it that way. For the special methods like __enter__ and __exit__, the tricky part isn't understanding what would happen once the methods were called; the tricky part is getting them to be called in the first place, if they were not declared inside the class or attached to the class. import types class Blank: pass foo = Blank() foo.name = foo1 foo.__exit__ = types.MethodType(lambda self, *args: print(self.name), foo) foo.__exit__() # works like a method in python3, prints foo1 with foo: pass -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
MRAB wrote: To give an analogy, it is like defining mammals as hairy animals which give birth to live young, which is correct for all mammals except for monotremes, which are mammals which lay eggs. Or the naked mole-rat. Or cetaceans (whales). The way I understand it, the main characteristic shared by all mammals is the presence of mammary glands in females. To wrest this back on topic, sometimes a definition can be improved without making it any more complicated. In the case of methods, perhaps instead of defined inside a class body it could say bound to a name in a class namespace. That's what really matters, not how it came to be there. -- Greg -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On 12/15/2011 12:01 AM, Steven D'Aprano wrote: On Wed, 14 Dec 2011 18:13:36 -0500, Terry Reedy wrote: On 12/14/2011 3:01 AM, Steven D'Aprano wrote: On Wed, 14 Dec 2011 01:29:13 -0500, Terry Reedy wrote: To complement what Eric says below: The with statement is looking for an instance *method*, which by definition, is a function attribute of a *class* (the class of the context manager) that takes an instance of the class as its first parameter. Note in the above that I am talking about *instance methods*. I'm not sure that is correct... I don't think that there is anything by definition about where methods live. Since you are disagreeing with my statement that *instance methods* are class attributes, you had better be claiming that *instance methods* can live elsewhere, else your statement has no sense. From the Python glossary: method: A function which is defined inside a class body. That is actually a bit too narrow, as a function can be added to the class after it is defined. But the point then is that it is treated as if defined inside the class body. Actually, I agree with Greg that rewording like he or I have suggested would be a good idea. So I'm happy for the glossary entry to stay as is, because complicating it would confuse the average coder more than it would educate them. But the definition as given is strictly wrong, because it fails to capture the essence of what distinguishes a method from other objects. It mistakes *how* you normally create a method for *what* a method is, a little like defining a hamburger is a foodstuff you get from McDonalds by giving them money. Here are three ways that the definition fails: (1) You can create a function inside a class, and it remains a function, so long as the class constructor (metaclass) never gets to build a method object from the function. It is easy to do: just hide it inside a wrapper object. class FunctionInsideClass(object): ... def func(x, y): # define a function inside a class ... return x + 2*y ... print(type(func)) # confirm is actually is a function ... attr = (func,) # hide it from the metaclass ... del func The function is not an attribute of the class, so my revised definition does not fail. class 'function' print(type(FunctionInsideClass.attr[0])) class 'function' (2) Instead of hiding the function from the metaclass, you can change the metaclass to something which doesn't make methods out of functions. I won't show an example, because it's tricky to get right (or at least *I* find metaclasses tricky). The default metaclass does not 'make methods out of functions'. Rather, functions that are attributes of an instance of 'type' are treated as methods *when accessed* via an instance of that instance. In Py 2, they were wrapped as unbound methods when accessed via the class, but no longer in Py 3, which simplifies things. Now, perhaps you can define a metaclass that disables method behavior, but practically everything one say about normal Python functioning goes out the window when one invokes 'metaclasses' (which do not even have to be classes!). So I do not consider this relevant to the discussion. (3) So the definition is too broad: you can have functions defined inside classes that are not methods. But it is also too narrow: you can have methods outside of classes. I'm not talking about bound and unbound methods, but about *creating* the method from scratch outside of a class. When you call the method constructor directly, you can create a method from a function defined outside of a class. def func(self, a): ... return self + a ... type(func) # Definitely a function. class 'function' obj = types.MethodType(func, 42) As I explained before, the intended input of MethodType is an *instance method* and an instance of the class it is an attribute of. (Or, I suspect, a class method and class, which is why the appropriate check in each case should be outside the call. But I am sticking with instance methods here.) If so, the output is a *bound method*. In your example above, func is not an instance method and obj is not a bound method. It is simply an partially evaluated curried function or if you prefer, a bound function. Take you pick, or make up your own term, but it is NOT an instance method, which is the subject under discussion. So obj has nothing to do with the definition of instance method and whether I had any authority for the definition I gave to the OP to help him solve his problem. obj(23) # Works as expected. 65 Yes, and I can think of three other ways to make an 'add42' function. type(obj) class 'method' So what? That means 'bound method', but since your input function to MethodType is not a method, its output is not a bound method. So there's a method which has never been inside a class, and couldn't even if you tried: int is a built-in immutable type. Calling it a 'method' when it is not even
Re: AttributeError in with statement (3.2.2)
On Thu, 15 Dec 2011 05:35:55 -0800, Steve Howell wrote: For the special methods like __enter__ and __exit__, the tricky part isn't understanding what would happen once the methods were called; the tricky part is getting them to be called in the first place, if they were not declared inside the class or attached to the class. If you *must* have per-instance special methods, my advice is to use a bit of scaffolding like this: class Whatever: def __enter__(self, *args): try: enter = self.__dict__['__enter__'] except KeyError: do_something_else() # or just let the exception occur else: enter(*args) Otherwise just live with the limitation that you can override all methods per-instance *except* dunders, and design your application accordingly. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Wed, 14 Dec 2011 01:29:13 -0500, Terry Reedy wrote: To complement what Eric says below: The with statement is looking for an instance *method*, which by definition, is a function attribute of a *class* (the class of the context manager) that takes an instance of the class as its first parameter. I'm not sure that is correct... I don't think that there is anything by definition about where methods live. Particularly not in Python where instance methods can be attributes of the instance itself. class Test(object): ... def method(self): ... print(This method is an attribute of the class.) ... t = Test() t.method() This method is an attribute of the class. import types t.method = types.MethodType( ... lambda self: print( ... This method is an attribute of the instance.), t) t.method() This method is an attribute of the instance. So the normal lookup rules that apply to data attributes, namely instance, then class, then superclasses, also applies to methods in Python. In languages that don't allow that sort of thing, like Java, you need to use convoluted design patterns like Dynamic Proxy to make it work. In Python, you just create a method and attach it on the instance. http://stackoverflow.com/questions/8260740/override-a-method-for-an- instance-of-a-class But this doesn't apply for special dunder attributes like __exit__, for speed reasons. (For new-style classes only, classic classes have no such special casing. This makes automatic delegation a breeze in Python 2 with classic classes, and a PITA in Python 3. Boo hiss.) -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
Steve Howell wrote: I'm using Python 3.2.2, and the following program gives me an error that I don't understand: class Foo: pass foo = Foo() foo.name = Steve def add_goodbye_function(obj): def goodbye(): print(goodbye + obj.name) obj.goodbye = goodbye add_goodbye_function(foo) foo.goodbye() # outputs goodbye Steve foo.__exit__ = foo.goodbye foo.__exit__() # outputs goodbye Steve with foo: # fails with AttributeError: __exit__ print(doing stuff) I am dynamically adding an attribute __exit__ to the variable foo, which works fine when I call it directly, but it fails when I try to use foo as the expression in the with statement. Here is the full output: python3 with.coffee goodbye Steve goodbye Steve Traceback (most recent call last): File with.coffee, line 17, in module with foo: # fails with AttributeError: AttributeError: __exit__ If you don't know it yet, contextlib.contextmanager offers a highlevel alternative: @contextmanager ... def goodbye(obj): ... try: ... yield obj ... finally: ... print(goodbye, obj.name) ... with goodbye(Foo()) as foo: ... print(hello, foo.name) ... hello Steve goodbye Steve -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Wednesday, December 14, 2011 4:01:24 PM UTC+8, Steven D#39;Aprano wrote: On Wed, 14 Dec 2011 01:29:13 -0500, Terry Reedy wrote: To complement what Eric says below: The with statement is looking for an instance *method*, which by definition, is a function attribute of a *class* (the class of the context manager) that takes an instance of the class as its first parameter. I'm not sure that is correct... I don't think that there is anything by definition about where methods live. Particularly not in Python where instance methods can be attributes of the instance itself. class Test(object): ... def method(self): ... print(This method is an attribute of the class.) ... t = Test() t.method() This method is an attribute of the class. import types t.method = types.MethodType( ... lambda self: print( ... This method is an attribute of the instance.), t) t.method() This method is an attribute of the instance. So the normal lookup rules that apply to data attributes, namely instance, then class, then superclasses, also applies to methods in Python. In languages that don't allow that sort of thing, like Java, you need to use convoluted design patterns like Dynamic Proxy to make it work. In Python, you just create a method and attach it on the instance. http://stackoverflow.com/questions/8260740/override-a-method-for-an- instance-of-a-class But this doesn't apply for special dunder attributes like __exit__, for speed reasons. (For new-style classes only, classic classes have no such special casing. This makes automatic delegation a breeze in Python 2 with classic classes, and a PITA in Python 3. Boo hiss.) -- Steven In Python an instance of an object of a class can have its own method. A living object can use those methods in the class definition and can acquire a new method at runtime. -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Thursday, December 15, 2011 12:08:32 AM UTC+8, 8 Dihedral wrote: On Wednesday, December 14, 2011 4:01:24 PM UTC+8, Steven D#39;Aprano wrote: On Wed, 14 Dec 2011 01:29:13 -0500, Terry Reedy wrote: To complement what Eric says below: The with statement is looking for an instance *method*, which by definition, is a function attribute of a *class* (the class of the context manager) that takes an instance of the class as its first parameter. I'm not sure that is correct... I don't think that there is anything by definition about where methods live. Particularly not in Python where instance methods can be attributes of the instance itself. class Test(object): ... def method(self): ... print(This method is an attribute of the class.) ... t = Test() t.method() This method is an attribute of the class. import types t.method = types.MethodType( ... lambda self: print( ... This method is an attribute of the instance.), t) t.method() This method is an attribute of the instance. So the normal lookup rules that apply to data attributes, namely instance, then class, then superclasses, also applies to methods in Python. In languages that don't allow that sort of thing, like Java, you need to use convoluted design patterns like Dynamic Proxy to make it work. In Python, you just create a method and attach it on the instance. http://stackoverflow.com/questions/8260740/override-a-method-for-an- instance-of-a-class But this doesn't apply for special dunder attributes like __exit__, for speed reasons. (For new-style classes only, classic classes have no such special casing. This makes automatic delegation a breeze in Python 2 with classic classes, and a PITA in Python 3. Boo hiss.) -- Steven In Python an instance of an object of a class can have its own method. A living object can use those methods in the class definition and can acquire a new method at runtime. Therefore, it is possible for an object to build its decision tree of actions toward a problem of various parameters in the run time. -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Tue, Dec 13, 2011 at 11:05 PM, Eric Snow ericsnowcurren...@gmail.com wrote: On Tue, Dec 13, 2011 at 10:42 PM, Steve Howell showel...@yahoo.com wrote: I'm using Python 3.2.2, and the following program gives me an error that I don't understand: class Foo: pass foo = Foo() foo.name = Steve def add_goodbye_function(obj): def goodbye(): print(goodbye + obj.name) obj.goodbye = goodbye add_goodbye_function(foo) foo.goodbye() # outputs goodbye Steve foo.__exit__ = foo.goodbye foo.__exit__() # outputs goodbye Steve with foo: # fails with AttributeError: __exit__ print(doing stuff) I am dynamically adding an attribute __exit__ to the variable foo, which works fine when I call it directly, but it fails when I try to use foo as the expression in the with statement. Here is the full output: python3 with.coffee goodbye Steve goodbye Steve Traceback (most recent call last): File with.coffee, line 17, in module with foo: # fails with AttributeError: AttributeError: __exit__ What am I doing wrong? That is a tricky one. As with many of the special methods (start and end with __) in Python, the underlying mechanism in the interpreter is directly pulling the function from the class object. It does not look to the instance object for the function at any time. See http://docs.python.org/reference/datamodel.html#special-method-lookup-for-new-style-classes. -eric Incidently, if you add the function directly to the class, everything works out: class Foo(object): # or class Foo: under Python 3 pass foo = Foo() foo.name = Steve def add_goodbye_function(cls): def goodbye(self, *args, **kwargs): print(goodbye + self.name) cls.goodbye = goodbye add_goodbye_function(type(foo)) foo.goodbye() # outputs goodbye Steve Foo.__exit__ = foo.goodbye foo.__exit__() # outputs goodbye Steve Foo.__enter__ = (lambda self: None) with foo: print(doing stuff) However, perhaps a better approach would be to put the different pieces directly into the class: class Foo: # python 3 def __init__(self, name): self.name = name def goodbye(self): print(goodbye + self.name) def __enter__(self): pass def __exit__(self, *args, **kwargs): self.goodbye() foo = Foo(Steve) foo.goodbye() # outputs goodbye Steve foo.__exit__() # outputs goodbye Steve with foo: print(doing stuff) If you want to be more dynamic about it you can do it, but it involves black magic. Chances are really good that being explicit through your class definition is the right approach. -eric -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Dec 14, 12:01 am, Steven D'Aprano steve
+comp.lang.pyt...@pearwood.info wrote:
[...]
So the normal lookup rules that apply to data attributes, namely
instance, then class, then superclasses, also applies to methods in
Python. In languages that don't allow that sort of thing, like Java, you
need to use convoluted design patterns like Dynamic Proxy to make it
work. In Python, you just create a method and attach it on the instance.
http://stackoverflow.com/questions/8260740/override-a-method-for-an-
instance-of-a-class
But this doesn't apply for special dunder attributes like __exit__, for
speed reasons. (For new-style classes only, classic classes have no such
special casing. This makes automatic delegation a breeze in Python 2 with
classic classes, and a PITA in Python 3. Boo hiss.)
Thanks to all who responded. Basically, I think the special casing of
the dunder attributes threw me off.
Typically, I would just build context managers from a class, but I
wanted to experiment with instances that were built up without the
standard Python class mechanisms, instead following a Javascript-like
closure-based object creation model.
This is what I came up with:
class WithWrapper:
def __init__(self, obj):
self.obj = obj
def __enter__(self):
self.obj['enter']()
def __exit__(self, *args):
self.obj['exit'](*args)
def greeter_context(hello, goodbye):
return {
'enter': lambda: print(---\n + hello),
'exit': lambda *args: print(goodbye)
}
gc_french = greeter_context(salut, a plus tard)
with WithWrapper(gc_french):
print(doing stuff)
gc_slang = greeter_context(yo, later)
with WithWrapper(gc_slang):
print(doing stuff)
--
http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On 12/15/2011 03:56 AM, Eric Snow wrote: On Tue, Dec 13, 2011 at 11:05 PM, Eric Snowericsnowcurren...@gmail.com wrote: If you want to be more dynamic about it you can do it, but it involves black magic. Chances are really good that being explicit through your class definition is the right approach. Note that the black spice is to use the __class__ attribute: foo.__class__.__exit__ = foo.goodbye -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Wed, Dec 14, 2011 at 12:14 PM, Lie Ryan lie.1...@gmail.com wrote: On 12/15/2011 03:56 AM, Eric Snow wrote: On Tue, Dec 13, 2011 at 11:05 PM, Eric Snowericsnowcurren...@gmail.com wrote: If you want to be more dynamic about it you can do it, but it involves black magic. Chances are really good that being explicit through your class definition is the right approach. Note that the black spice is to use the __class__ attribute: foo.__class__.__exit__ = foo.goodbye Good point. 'type(foo).__exit__ ...' might be even better. Regardless, to get the dynamicism to which Steve originally referred (make it work when added directly to the instance), you have to use a metaclass, which is black magic[1]. However, rarely is that sort of thing needed, so the relatively superfluous details would likely only cloud the question at hand. -eric [1] I will point out that metaclasses aren't really all that bad. They are a tool worth understanding, even if you don't use them all the time. Understanding them also opens up a whole new world of understanding how Python works, particularly regarding name lookup. -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On 12/14/2011 3:01 AM, Steven D'Aprano wrote: On Wed, 14 Dec 2011 01:29:13 -0500, Terry Reedy wrote: To complement what Eric says below: The with statement is looking for an instance *method*, which by definition, is a function attribute of a *class* (the class of the context manager) that takes an instance of the class as its first parameter. I'm not sure that is correct... I don't think that there is anything by definition about where methods live. From the Python glossary: method: A function which is defined inside a class body. That is actually a bit too narrow, as a function can be added to the class after it is defined. But the point then is that it is treated as if defined inside the class body. Particularly not in Python where instance methods can be attributes of the instance itself. This is access, not definition or actual location. The glossary entry go on to say: If called as an attribute of an instance of that class, the method will get the instance object as its first argument (which is usually called self). This does *not* happen if a callable is found in the instance-specific dictionary. An instance method is a function (callable) attribute of a class that gets special treatment when accessed (indirectly) through an instance of that class (or subclass thereof). class Test(object): ... def method(self): ... print(This method is an attribute of the class.) ... t = Test() t.method() This method is an attribute of the class. The bound method t.method is an instance the class exposed as types.MethodType. In other words, isinstance(t.method, types.MethodType) == True import types t.method = types.MethodType( ... lambda self: print( ... This method is an attribute of the instance.), t) Calling any old fruit an apple does not make it one. Calling any old function a method does not make it one. 'types.MethodType' is the exposed name of the class the interpreter uses to create bound methods from a method and an instance of the class containing the method. I believe the interpreter does an isinstance check, but it must do that before calling the class, and not in the bound method constructor itself. In any case, a bound method is not a method. So the printed statement is not true. In this case, the result is not really even a bound method, as the function argument is not a method, so we cannot even ask if the second arg is an instance of the function class container. MethodType is a special case of functools.partial, which was added later. You could have used the latter to the same effect. Or you could have used any old function that printed the same thing. There is no relation between the object passed as the second arg of MethodType and what you do with the resulting callable. Either 't' could be something else. See below. t.method() This method is an attribute of the instance. Yes, the callable (which is not a method) is (currently) an attribute of the instance. But that is irrelevant to its operation. t.method is just a callable, in particular, a pseudo bound method, not a method. It is *not* supplying the instance it is called on as the first parameter of the callable. The arguemnt (which is not used) has already been supplied. These produce the same output: class B: pass b = B() b.method = t.method b.method() f = t.method f() t.method = lambda: print(This method is an attribute of the instance.) t.method() So the normal lookup rules that apply to data attributes, namely instance, then class, then superclasses, also applies to methods in Python. When you ask the interpreter to resolve a.x, x is just a supposed attribute, and the interpreter has no idea what class the result should be. But this doesn't apply for special dunder attributes like __exit__, for speed reasons. It does not apply to dunder *methods* because they are reserved names defined to be (bound to) methods. So the interpreter knowing that it is looking for a method and that methods have to be attributes of classes, goes directly to the class. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Wed, 14 Dec 2011 18:13:36 -0500, Terry Reedy wrote: On 12/14/2011 3:01 AM, Steven D'Aprano wrote: On Wed, 14 Dec 2011 01:29:13 -0500, Terry Reedy wrote: To complement what Eric says below: The with statement is looking for an instance *method*, which by definition, is a function attribute of a *class* (the class of the context manager) that takes an instance of the class as its first parameter. I'm not sure that is correct... I don't think that there is anything by definition about where methods live. From the Python glossary: method: A function which is defined inside a class body. That is actually a bit too narrow, as a function can be added to the class after it is defined. But the point then is that it is treated as if defined inside the class body. First off, let me preface this by saying that I'm not necessarily saying that the above glossary definition needs to be changed. For most purposes, it is fine, since *nearly always* methods are created as functions defined inside the class body. But it needs to be understood in context as a simplified, hand-wavy definition which covers 99% of the common cases, and not a precise, definitive technical definition. To give an analogy, it is like defining mammals as hairy animals which give birth to live young, which is correct for all mammals except for monotremes, which are mammals which lay eggs. So I'm happy for the glossary entry to stay as is, because complicating it would confuse the average coder more than it would educate them. But the definition as given is strictly wrong, because it fails to capture the essence of what distinguishes a method from other objects. It mistakes *how* you normally create a method for *what* a method is, a little like defining a hamburger is a foodstuff you get from McDonalds by giving them money. Here are three ways that the definition fails: (1) You can create a function inside a class, and it remains a function, so long as the class constructor (metaclass) never gets to build a method object from the function. It is easy to do: just hide it inside a wrapper object. class FunctionInsideClass(object): ... def func(x, y): # define a function inside a class ... return x + 2*y ... print(type(func)) # confirm is actually is a function ... attr = (func,) # hide it from the metaclass ... del func ... class 'function' print(type(FunctionInsideClass.attr[0])) class 'function' (2) Instead of hiding the function from the metaclass, you can change the metaclass to something which doesn't make methods out of functions. I won't show an example, because it's tricky to get right (or at least *I* find metaclasses tricky). (3) So the definition is too broad: you can have functions defined inside classes that are not methods. But it is also too narrow: you can have methods outside of classes. I'm not talking about bound and unbound methods, but about *creating* the method from scratch outside of a class. When you call the method constructor directly, you can create a method from a function defined outside of a class. def func(self, a): ... return self + a ... type(func) # Definitely a function. class 'function' obj = types.MethodType(func, 42) obj(23) # Works as expected. 65 type(obj) class 'method' So there's a method which has never been inside a class, and couldn't even if you tried: int is a built-in immutable type. So what are methods? In Python, methods are wrappers around functions which automatically pass the instance to the inner function object. Under normal circumstances, you create methods by declaring functions inside a class, but that's not the only way to create methods, and it is not the only place they can be found. Note that this definition can easily be adapted to describe class methods and static methods too: class methods are wrappers around functions that automatically pass the class to the inner function, and static methods are wrappers which don't pass any automatic arguments. Particularly not in Python where instance methods can be attributes of the instance itself. This is access, not definition or actual location. Not so. In the example I gave, the method *really is* inside the instance, stored in the instance __dict__ and not the class __dict__. The glossary entry go on to say: If called as an attribute of an instance of that class, the method will get the instance object as its first argument (which is usually called self). This does *not* happen if a callable is found in the instance-specific dictionary. That's right. Methods are special not because of where they are, but because of what they are. An instance method is a function (callable) attribute of a class that gets special treatment when accessed (indirectly) through an instance of that class (or subclass thereof). Methods aren't functions at all, not in the isinstance sense. They are functions in the sense that
Re: AttributeError in with statement (3.2.2)
On 15/12/2011 05:01, Steven D'Aprano wrote: On Wed, 14 Dec 2011 18:13:36 -0500, Terry Reedy wrote: On 12/14/2011 3:01 AM, Steven D'Aprano wrote: On Wed, 14 Dec 2011 01:29:13 -0500, Terry Reedy wrote: To complement what Eric says below: The with statement is looking for an instance *method*, which by definition, is a function attribute of a *class* (the class of the context manager) that takes an instance of the class as its first parameter. I'm not sure that is correct... I don't think that there is anything by definition about where methods live. From the Python glossary: method: A function which is defined inside a class body. That is actually a bit too narrow, as a function can be added to the class after it is defined. But the point then is that it is treated as if defined inside the class body. First off, let me preface this by saying that I'm not necessarily saying that the above glossary definition needs to be changed. For most purposes, it is fine, since *nearly always* methods are created as functions defined inside the class body. But it needs to be understood in context as a simplified, hand-wavy definition which covers 99% of the common cases, and not a precise, definitive technical definition. To give an analogy, it is like defining mammals as hairy animals which give birth to live young, which is correct for all mammals except for monotremes, which are mammals which lay eggs. [snip] Or the naked mole-rat. Or cetaceans (whales). -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Thu, 15 Dec 2011 05:15:58 +, MRAB wrote: On 15/12/2011 05:01, Steven D'Aprano wrote: On Wed, 14 Dec 2011 18:13:36 -0500, Terry Reedy wrote: On 12/14/2011 3:01 AM, Steven D'Aprano wrote: On Wed, 14 Dec 2011 01:29:13 -0500, Terry Reedy wrote: To complement what Eric says below: The with statement is looking for an instance *method*, which by definition, is a function attribute of a *class* (the class of the context manager) that takes an instance of the class as its first parameter. I'm not sure that is correct... I don't think that there is anything by definition about where methods live. From the Python glossary: method: A function which is defined inside a class body. That is actually a bit too narrow, as a function can be added to the class after it is defined. But the point then is that it is treated as if defined inside the class body. First off, let me preface this by saying that I'm not necessarily saying that the above glossary definition needs to be changed. For most purposes, it is fine, since *nearly always* methods are created as functions defined inside the class body. But it needs to be understood in context as a simplified, hand-wavy definition which covers 99% of the common cases, and not a precise, definitive technical definition. To give an analogy, it is like defining mammals as hairy animals which give birth to live young, which is correct for all mammals except for monotremes, which are mammals which lay eggs. [snip] Or the naked mole-rat. Or cetaceans (whales). Naked mole-rats and cetaceans do have hair, just not very much of it. E.g. http://marinelife.about.com/od/cetaceans/f/whaleshair.htm -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Thu, 15 Dec 2011 05:01:21 +, Steven D'Aprano wrote: From the Python glossary: method: A function which is defined inside a class body. That is actually a bit too narrow, as a function can be added to the class after it is defined. But the point then is that it is treated as if defined inside the class body. [...] But the definition as given is strictly wrong, because it fails to capture the essence of what distinguishes a method from other objects. [...] TL;DR http://users.rcn.com/python/download/Descriptor.htm#functions-and-methods Short and sweet. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
AttributeError in with statement (3.2.2)
I'm using Python 3.2.2, and the following program gives me an error that I don't understand: class Foo: pass foo = Foo() foo.name = Steve def add_goodbye_function(obj): def goodbye(): print(goodbye + obj.name) obj.goodbye = goodbye add_goodbye_function(foo) foo.goodbye() # outputs goodbye Steve foo.__exit__ = foo.goodbye foo.__exit__() # outputs goodbye Steve with foo: # fails with AttributeError: __exit__ print(doing stuff) I am dynamically adding an attribute __exit__ to the variable foo, which works fine when I call it directly, but it fails when I try to use foo as the expression in the with statement. Here is the full output: python3 with.coffee goodbye Steve goodbye Steve Traceback (most recent call last): File with.coffee, line 17, in module with foo: # fails with AttributeError: AttributeError: __exit__ What am I doing wrong? -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On Tue, Dec 13, 2011 at 10:42 PM, Steve Howell showel...@yahoo.com wrote: I'm using Python 3.2.2, and the following program gives me an error that I don't understand: class Foo: pass foo = Foo() foo.name = Steve def add_goodbye_function(obj): def goodbye(): print(goodbye + obj.name) obj.goodbye = goodbye add_goodbye_function(foo) foo.goodbye() # outputs goodbye Steve foo.__exit__ = foo.goodbye foo.__exit__() # outputs goodbye Steve with foo: # fails with AttributeError: __exit__ print(doing stuff) I am dynamically adding an attribute __exit__ to the variable foo, which works fine when I call it directly, but it fails when I try to use foo as the expression in the with statement. Here is the full output: python3 with.coffee goodbye Steve goodbye Steve Traceback (most recent call last): File with.coffee, line 17, in module with foo: # fails with AttributeError: AttributeError: __exit__ What am I doing wrong? That is a tricky one. As with many of the special methods (start and end with __) in Python, the underlying mechanism in the interpreter is directly pulling the function from the class object. It does not look to the instance object for the function at any time. See http://docs.python.org/reference/datamodel.html#special-method-lookup-for-new-style-classes. -eric -- http://mail.python.org/mailman/listinfo/python-list
Re: AttributeError in with statement (3.2.2)
On 12/14/2011 1:05 AM, Eric Snow wrote: On Tue, Dec 13, 2011 at 10:42 PM, Steve Howellshowel...@yahoo.com wrote: I'm using Python 3.2.2, and the following program gives me an error that I don't understand: class Foo: pass foo = Foo() foo.name = Steve def add_goodbye_function(obj): def goodbye(): print(goodbye + obj.name) obj.goodbye = goodbye add_goodbye_function(foo) foo.goodbye() # outputs goodbye Steve foo.__exit__ = foo.goodbye foo.__exit__() # outputs goodbye Steve foo.goodbye, aliased as foo.__exit__, is a *function* attribute of the foo *instance*. with foo: # fails with AttributeError: __exit__ print(doing stuff) I am dynamically adding an attribute __exit__ to the variable foo, which works fine when I call it directly, but it fails when I try to use foo as the expression in the with statement. Here is the full output: python3 with.coffee goodbye Steve goodbye Steve Traceback (most recent call last): File with.coffee, line 17, inmodule with foo: # fails with AttributeError: AttributeError: __exit__ What am I doing wrong? To complement what Eric says below: The with statement is looking for an instance *method*, which by definition, is a function attribute of a *class* (the class of the context manager) that takes an instance of the class as its first parameter. Notice that goodbye does not take any parameter. The first parameter is conventionally called 'self', but that is not a requirement. That it be there, that it expect to be bound to an instance, and that the function be bound to the class are requirements. That is a tricky one. As with many of the special methods (start and end with __) in Python, the underlying mechanism in the interpreter is directly pulling the function from the class object. It does not look to the instance object for the function at any time. See http://docs.python.org/reference/datamodel.html#special-method-lookup-for-new-style-classes. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
