Re: [Python-Dev] Class decorators can't be pickled, which breaks multiprocessing and concurrent.futures. Any plans of improving this?
11.08.18 23:08, Santiago Basulto пише:
Hello folks! I'm using the `concurrent.futures.ProcessPoolExecutor` with
a couple of functions that have been decorated with a class decorator.
Both `concurrent.futures` and `multiprocessing` breaks because "the
object's can't be pickled". There's a really simple fix for this, which
is just, instead of "decorating" the function (with the @), instantiate
the decorator and use it directly.
Example. This is my (very simple, for demonstration purposes) decorator:
class CheckOnlyIntegers:
def __init__(self, fn):
self.fn = fn
def __call__(self, *args):
if not all([type(arg) == int for arg in args]):
raise ValueError("Invalid param is not an integer")
return self.fn(*args)
If I define a simple `add` function and decorate it using the
`CheckOnlyIntegers` decorator:
@CheckOnlyIntegers
def add(x, y):
return x + y
and try using a regular `ProcessPoolExecutor().submit(add, 2, 3)`, it
fails with:
```
Can't pickle : it's not the same object as
__main__.add.
```
By default instances of Python classes are pickled by pickling a set of
their attributes. Functions are pickled by name. But since the name of
self.fn corresponds to the decorated function, not the function itself,
it can't be pickled.
You can implement the explicit pickle support for your decorator that
bypass this limitation.
def __reduce__(self):
return self.fn.__qualname__
Now the decorated function will be pickled by name.
It may help to set also self.__module__ = self.fn.__module__ in the
constructor.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Class decorators can't be pickled, which breaks multiprocessing and concurrent.futures. Any plans of improving this?
Hello folks! I'm using the `concurrent.futures.ProcessPoolExecutor` with a
couple of functions that have been decorated with a class decorator. Both
`concurrent.futures` and `multiprocessing` breaks because "the object's
can't be pickled". There's a really simple fix for this, which is just,
instead of "decorating" the function (with the @), instantiate the
decorator and use it directly.
Example. This is my (very simple, for demonstration purposes) decorator:
class CheckOnlyIntegers:
def __init__(self, fn):
self.fn = fn
def __call__(self, *args):
if not all([type(arg) == int for arg in args]):
raise ValueError("Invalid param is not an integer")
return self.fn(*args)
If I define a simple `add` function and decorate it using the
`CheckOnlyIntegers` decorator:
@CheckOnlyIntegers
def add(x, y):
return x + y
and try using a regular `ProcessPoolExecutor().submit(add, 2, 3)`, it fails
with:
```
Can't pickle : it's not the same object as
__main__.add.
```
The fix for this is simple, instead of "decorating" the function,
instantiate the class and use a different name:
def add(x, y):
return x + y
add_2 = CheckOnlyIntegers(add)
In this case `ProcessPoolExecutor().submit(add_2, 2, 3)` works correctly.
(here's the full sample code
)
I know this is an issue with the pickle module (not concurrent.futures or
multiprocessing). But are there any ways of improving this in future
versions? Not being able to pickle functions decorated with Class
Decorators seems like an unnecessary limitation.
Thanks for your feedback!
--
Santiago Basulto.-
Up!
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
In the discussion over class decorators, Jim Jewett writes: I have often started with a function, and ended up replacing it with a callable object so that I could save state without resorting to defalt args or worse. I would prefer to decorate these exactly like the functions they replace. I have observed the entire discussion about class decorators with absolutely no opinion, until I read Jim's brief post quoted above. I am now completely convinced that class decorators ought to exist and behave exactly like function decorators. Thanks, Jim for pointing out what should have been obvious to me from the start. The ability to use callable objects as functions is a powerful tool in Python, and ought not be broken by decorator inconsistencies. -- Michael Chermside ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Michael Chermside wrote: In the discussion over class decorators, Jim Jewett writes: I have often started with a function, and ended up replacing it with a callable object so that I could save state without resorting to defalt args or worse. I would prefer to decorate these exactly like the functions they replace. I have observed the entire discussion about class decorators with absolutely no opinion, until I read Jim's brief post quoted above. I am now completely convinced that class decorators ought to exist and behave exactly like function decorators. Thanks, Jim for pointing out what should have been obvious to me from the start. The ability to use callable objects as functions is a powerful tool in Python, and ought not be broken by decorator inconsistencies. While I agree with you, I don't think this conclusion is as obviously correct as it might first appear, because you don't want to decorate the class itself in such cases. You don't even want to decorate the class's __call__ method - you want to decorate an *instance* of the class, as that is what will mimic the interface of the original function. Compare: Py def f(): ...print Hi World from something! ... Py f() Hi World from something! and: Py class f(object): ... def __call__(self): ... print Hi world from %s! % self ... Py f() __main__.f object at 0x00AE1F70 Py f()() Hi world from __main__.f object at 0x00AE7130! Clearly, these two definitions of 'f' are _not_ equivalent - the first one is a callable, but the latter is a callable factory. Applying the original function's decorators to the class or its __call__ method will yield nonsense. To get an object from the second approach with an interface equivalent to the original f (only with an automatically supplied mutable data store as its first argument), you instead want to write: Py class f(object): ... def __call__(self): ... print Hi world from %s! % self ... Py f = f() Py f() Hi world from __main__.f object at 0x00AE7190! If the original f had decorators applied to it, then you would have to apply those to the final resulting bound method in the example, not to the class definition. OTOH, all is not lost, as a simple class decorator would allow Jim's use case to be satisfied quite handily: def instance(cls): return cls() Then: @deco1 @deco2 def f(): pass Could easily be replaced with: @deco1 @deco2 @instance class f(object): def __call__(self): pass Cheers, Nick. P.S. If all you want is somewhere to store mutable state between invocations, you can always use the function's own attribute space: Py def f(): ... print Hi world from %s! % f ... Py f() Hi world from function f at 0x00AE90B0! -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 04:47 AM 3/31/2006 -0800, Michael Chermside wrote: In the discussion over class decorators, Jim Jewett writes: I have often started with a function, and ended up replacing it with a callable object so that I could save state without resorting to defalt args or worse. I would prefer to decorate these exactly like the functions they replace. I have observed the entire discussion about class decorators with absolutely no opinion, until I read Jim's brief post quoted above. I am now completely convinced that class decorators ought to exist and behave exactly like function decorators. Thanks, Jim for pointing out what should have been obvious to me from the start. The ability to use callable objects as functions is a powerful tool in Python, and ought not be broken by decorator inconsistencies. Unless the class has a metaclass implementing __call__, or you mean that you want instance creation to be a call, I don't understand what you mean. Nonetheless, the discussion has only been about *where* the decorators go and what syntax they use. Nobody has proposed any change in decorator semantics, so please stop attacking this meaningless strawman. Moving from: @foo def bar(...): ... to: class bar: @class foo def __init___(...): ... instead of: @foo class bar: def __init___(...): ... is a trivial difference in editing: you type class one more time. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On Friday 31 March 2006 11:52, Phillip J. Eby wrote: class bar: @class foo def __init___(...): ... The more I think about it, the more I like the @class foo syntax. The existing syntax for functions doesn't have anything between the decorators and the def; having class decorators embedded within the class should still allow the docstring to be the first thing, so there's more distance between the decorator and the name being decorated. The extra hint about what's being decorated is nice. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Class decorators
Nick Coghlan wrote: [ much good, including the @instance decorator ] P.S. If all you want is somewhere to store mutable state between invocations, you can always use the function's own attribute space def f(): print Hi world from %s! % f f() Hi world from function f at 0x00AE90B0! Not really. That assumes the expected name is (permanently) bound to *this* function in this function's globals. That's normally true, but not always, so relying on it seems wrong. f=a string g() Hi world from a string! And of course, sometimes I really do want shared state, or helpers (like other methods on a class), or one-time ininitialization plus per-call parameters (like the send method on 2.5 generators), or ... -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Jim Jewett wrote: Nick Coghlan wrote: [ much good, including the @instance decorator ] P.S. If all you want is somewhere to store mutable state between invocations, you can always use the function's own attribute space def f(): print Hi world from %s! % f f() Hi world from function f at 0x00AE90B0! Not really. That assumes the expected name is (permanently) bound to *this* function in this function's globals. That's normally true, but not always, so relying on it seems wrong. Well, true. If you want it to be bullet-proof, you have to use a closure instead: def make_f(): ... def f(): ... print Hi world from %s! % f ... return f ... f = make_f() f() Hi world from function f at 0x00AE90B0! The point about the decorators still holds - they stay with the function they're decorating (the inner one), and you don't need to make any of the changes needed in order to decorate the class instead. The above is also a use case for a *function* decorator I've occasionally wanted - an @result decorator, that is the equivalent of the @instance decorator I suggested for classes: def result(f): ... return f() ... @result ... def f(): ... def f(): ... print Hi world from %s! % f ... return f ... f() Hi world from function f at 0x00AE90B0! I never actually did it, though, as I was stuck on Python 2.2 at the time. This is something of a tangent to the real discussion though, so I'll leave this one there :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Phillip J. Eby wrote: Are you actually *using* this IOClass thing, or is this just a hypothetical proposal? I'm using it. It's not hypothetical. Putting all the info I want in the decorator itself wouldn't be very nice in my case, or at least that's my opinion. One issue is that I'm also abusing a class statement to create a namespace for my extra info. Inserting that as a decorator argument obviously wouldn't be possible... -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
[Jack Diederich] Classes have a unique property in that they are the easiest way to make little namespaces in python. [Greg Ewing] For a while now, I've been wondering whether it would be worth having a construct purely for creating little namespaces, instead of abusing a class for this. FWIW, I do not consider it an abuse to use a class to create a small namespace. Essentially that is what it is for -- it matters not whether the class has no methods. What I've been wanting is the ability to easily define functions directly into an existing namespace: class A: pass def A.mymethod(x): return x+1 The latter definition is equivalent to: A.mymethod = lambda(x): x+1 This ability to inject function definitions into an existing namespace is the one missing piece to enable clean use of Protocol OO techniques: a = Object.copy() b = a.copy() def b.f(x): return x+1 c = b.copy() def c.f(x): return x+2 Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
[EMAIL PROTECTED] wrote on 30/03/2006 11:38:30: Jack Diederich wrote: Classes have a unique property in that they are the easiest way to make little namespaces in python. For a while now, I've been wondering whether it would be worth having a construct purely for creating little namespaces, instead of abusing a class for this. I've been thinking about an 'instance' statement that creates an instance of a class: instance my_thing(MyClass): # attribute assignments go here Maybe this would be a use for the proposal a while back where: 'statement' name(args): ... implied name = 'statement'(name, args, namespace) then we could have: namespace foo: ... and interface IFoo(IBar): ... and your instance my_thing(MyClass): ... ? Cheers, Ben -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python- dev/python%40theyoungfamily.co.uk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Raymond Hettinger wrote: FWIW, I do not consider it an abuse to use a class to create a small namespace. Essentially that is what it is for -- it matters not whether the class has no methods. Two problems with that: * The word class in front of it is a misnomer if you've no intention of using it as a class. * It's not a completely blank slate -- it comes with various baggage in the form of __xxx__ attributes (my registration function has to filter those out). Plus some unwanted behaviour such as being callable, and doing magic things with attribute access when the attribute is a descriptor. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
[EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote on 30/03/2006 11:38:30: Jack Diederich wrote: Classes have a unique property in that they are the easiest way to make little namespaces in python. For a while now, I've been wondering whether it would be worth having a construct purely for creating little namespaces, instead of abusing a class for this. I've been thinking about an 'instance' statement that creates an instance of a class: instance my_thing(MyClass): # attribute assignments go here Maybe this would be a use for the proposal a while back where: 'statement' name(args): ... implied name = 'statement'(name, args, namespace) [...] I like that generalization (since a class definition statement currently does about the same anyway). However, please post that to the python-3000 list as this would be a change for Python 3. Cheers, Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Fred L. Drake, Jr. wrote: It's too bad this syntax is ambiguous: class Foo: Docstring here, blah blah blah @implements(IFoo) As this achieves a desirable highlighting of the specialness, without forcing the decorator outside the class. Oh well. Agreed, but... guess we can't have everything. On the other hand, something like: class Foo: Documentation is good. @class implements(IFoo) is not ambiguous. Hmm. It even says what it means. :-) This is quite reminiscent of Ruby to me, where: class Foo: implements(IFoo) basically means: class Foo: pass Foo.implements(IFoo) For a variety of reasons that doesn't work for Python, but what you propose accomplishes the same basic thing. I'm coming in a little late on all this, but I find moving the decorator inside the class statement to be a substantial improvement, even if it is also a trivial improvement ;) Anytime I've done thought experiments about using class decorators, the results is very hard to read. That classes are inherently declarative and open, while functions are imperative and closed, makes the constructs very different. -- Ian Bicking / [EMAIL PROTECTED] / http://blog.ianbicking.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Class decorators
Phillip J. Eby wrote: I don't even recall seeing any examples of class decorators being used without arguments! I have often started with a function, and ended up replacing it with a callable object so that I could save state without resorting to defalt args or worse. I would prefer to decorate these exactly like the functions they replace. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On Wed, 2006-03-29 at 00:01 -0500, Phillip J. Eby wrote: For some reason, this doesn't bother me with functions. But then, I can't remember how often I've actually needed to use two decorators on the same function, or how many times a function decorator's arguments took multiple lines to list. Both of these are routine occurrences for my class use cases. We have a couple of instances where we use multiple decorators, and even a case where one of them takes an argument. It's not too bad. I agree with your comments about class decorator readability though. -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On 3/29/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 11:35 PM 3/28/2006 -0500, Fred L. Drake, Jr. wrote:For Zope 3, we have decorators that work with the component architecture (I'msure Phillip is familiar with these).They're used with functions toindicate that the function adapts a particular kind of object, or that it implements or provides a particular interface.We have different functionsthat get used for this purpose in classes that are executed within the bodyof the class.There's some merit to being able to use a single set of functions in both cases, since the use cases are the same.I'm not sure I'dwant to change the existing pattern, though, since it's already so widespreadwithin the Zope 3 codebase (including 3rd-party components). If we're using Zope 3 as an example, I personally find that: class Foo: Docstring here, blah blah blah implements(IFoo)is easier to read than: @implements(IFoo) class Foo: Docstring here, blah blah blah Yeah, but in the first case implements(IFoo) has to do dirty hacks involving sys.getframe(), so you win in once place but lose in the other one. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On Wed, Mar 29, 2006 at 01:11:06AM -0500, Fred L. Drake, Jr. wrote: On Wednesday 29 March 2006 00:48, Fred L. Drake, Jr. wrote: I think the existing usage for classes is perfectly readable. The @-syntax works well for functions as well. On re-reading what I wrote, I don't think I actually clarified the point I was trying to make originally. My point wasn't that I desparately need @-syntax for class decorators (I don't), or see it as inherantly superior in some way. It's much simpler than that: I just want to be able to use the same syntax for a group of use cases regardless of whether the target is a function or a class. This fits into the nice-to-have category for me, since the use case can be the same regardless of whether I'm decorating a class or a function. (I will note that when this use case applies to a function, it's usually a module-level function I'm decorating rather than a method.) Agreed, let's not have the decorator syntax argument all over again. Once someone knows how a function decorator works they should be able to guess how a class decorator works. In my old patch[1] the grammar production for decorators was: decorated_thing: decorators (funcdef|classdef) Which makes sense, once you know how to decorate one thing you know how to decorate all things. -jackdied [1] http://python.org/sf/1007991 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On 3/28/06, Phillip J. Eby [EMAIL PROTECTED] wrote: If we're using Zope 3 as an example, I personally find that: class Foo: Docstring here, blah blah blah implements(IFoo) is easier to read than: @implements(IFoo) class Foo: Docstring here, blah blah blah But the former also smells more of magic. A function putting a magic variable into the scope in which it is called isn't exactly elegant use of Python. You've probably read too much Zope code to be sensitive to this argument; but to me it still grates. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 10:42 AM 3/30/2006 +1200, Greg Ewing wrote: Fred L. Drake, Jr. wrote: class Foo: Documentation is good. @class implements(IFoo) That's an interesting idea. It could be applied to functions, too: def myfunc(myargs): Documentation is hoopy @def biguglydecorator(longconvolutedarglist) -1; there should be only one obvious way to do it. Plus, @class is a noun and reads like it's saying something about the class; @def doesn't. It's too verby. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 11:07 AM 3/29/2006 -0800, Guido van Rossum wrote:
On 3/28/06, Phillip J. Eby [EMAIL PROTECTED] wrote:
If we're using Zope 3 as an example, I personally find that:
class Foo:
Docstring here, blah blah blah
implements(IFoo)
is easier to read than:
@implements(IFoo)
class Foo:
Docstring here, blah blah blah
But the former also smells more of magic.
My comment above was only about readable *placement* of the decorators, not
the actual syntax. Many approaches to the actual syntax in the body are
possible.
For example, what did you think of Fred Drakes's @class proposal? To
specify it formally, one could say that this:
@class EXPR
in a class scope would expand to the equivalent of:
locals().setdefault('__decorators__',[]).append(EXPR)
and is a syntax error if placed anywhere else. That, combined with support
for processing __decorators__ at class creation time, would fulfill the
desired semantics without any implicit magic.
(The locals() part could of course be implemented in bytecode as
LOAD_LOCALS, since class scopes implement their locals as a
dictionary. That would avoid the need for adding any new bytecodes, since
this isn't a performance-sensitive feature.)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On Wed, Mar 29, 2006 at 07:23:03PM -0500, Phillip J. Eby wrote:
At 11:07 AM 3/29/2006 -0800, Guido van Rossum wrote:
On 3/28/06, Phillip J. Eby [EMAIL PROTECTED] wrote:
If we're using Zope 3 as an example, I personally find that:
class Foo:
Docstring here, blah blah blah
implements(IFoo)
is easier to read than:
@implements(IFoo)
class Foo:
Docstring here, blah blah blah
But the former also smells more of magic.
My comment above was only about readable *placement* of the decorators, not
the actual syntax. Many approaches to the actual syntax in the body are
possible.
For example, what did you think of Fred Drakes's @class proposal? To
specify it formally, one could say that this:
@class EXPR
in a class scope would expand to the equivalent of:
locals().setdefault('__decorators__',[]).append(EXPR)
and is a syntax error if placed anywhere else. That, combined with support
for processing __decorators__ at class creation time, would fulfill the
desired semantics without any implicit magic.
A function decorator takes a function as an argument and returns something
(probably a function and maybe even the very same function).
This is exactly what class decorators should do or we should call them
something else and give them a distinct syntax.
A function decorator is there to replace code like:
def myfunc(a, b, c):
# half a screen of code
myfunc = mangle(myfunc)
Likewise class decorators would save me from typing
class MyClass:
# many functions taking half a screen of code each
register(MyClass, db_id=20)
I used to do this with metaclasses but stopped because it requires making
'db_id' a member of the class which is magically noticed by a metaclass
two files away. Using metaclasses also required gross hacks like checking
for a 'DO_NOT_REGISTER' member for subclasses that wanted to inherit from
a class that had a Register metaclass but didn't want to be registered.
Yuck.
If you want to do lots of Zopeish stuff mostly inside the class write
a decorator that looks for it in the class body.
@zope.find_magic_in_attr('mymagic')
class MyClass:
mymagic = [] # some long hairy thing
Talking about something other than a decorator or proposing all new
syntax is just going to get this pronounced out of existence.
If-I-wanted-zope-I'd-use-zope-ly,
-jackdied
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 08:00 PM 3/29/2006 -0500, Jack Diederich wrote: A function decorator takes a function as an argument and returns something (probably a function and maybe even the very same function). So would class decorators. This is exactly what class decorators should do or we should call them something else and give them a distinct syntax. Yep. A function decorator is there to replace code like: def myfunc(a, b, c): # half a screen of code myfunc = mangle(myfunc) Likewise class decorators would save me from typing class MyClass: # many functions taking half a screen of code each register(MyClass, db_id=20) Yep. Talking about something other than a decorator You lost me there. Nobody's argued (AFAIK) for class decorators being anything other than single-argument functions that take a class as input and return a class as output. or proposing all new syntax is just going to get this pronounced out of existence. Which wouldn't be a bad thing, IMO. There's no point in adding them if they're not an actual improvement over what we can do now. Comments about the hackishness of the implementation used by Zope and PEAK are also off-base; nobody proposed that everyone should go off and implement their own such hacks! Meanwhile, inclusion of such a facility in the stdlib isn't without precedent; there are IIRC at least *6* sys._getframe hacks in the stdlib already. Nothing stops us from adding a 'decorate_class' function to the stdlib that's used like this, for example: class MyClass: decorate_class( register(db_id=20), implements(IFoo), ... ) or to just make the class machinery interpret a __decorators__ attribute (a variant on an old suggestion of Guido's): class MyClass: __decorators__ = [register(db_id=20), implements(IFoo)] The only *implementation* reason to give this special syntax, IIUC, is to allow implementations like Jython and IronPython and Pyrex to statically recognize certain decorators at compile-time. There are of course also non-implementation reasons to have class decorator syntax (such as EIBTI), and I agree with that. But readability also counts, and the readability of @decorators on the outside of a class tends to suck as the number of decorators and arguments increases. What's more, I haven't seen anybody posting any counterexamples to show that it doesn't suck for common use cases. Indeed, at the moment I don't even recall seeing any examples of class decorators being used without arguments! I also suspect that any actual Jython/IronPython examples are likely to be at least as verbose as Zope and PEAK's, and probably more likely to include multiple decorators. (Depending on how Java annotations and .Net attribs would be translated to decorators.) So, I'm personally not in favor of adding class decorators with a syntax that blindly imitates that of function decorators, without a proper examination of the use cases. This is precisely the argument that Guido used to veto class decorators in 2.4, and nothing about the issue has changed since then, except for the subject being reopened to consideration. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Phillip J. Eby wrote: My comment above was only about readable *placement* of the decorators, not the actual syntax. The placement is part of the syntax... -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Phillip J. Eby wrote: the readability of @decorators on the outside of a class tends to suck as the number of decorators and arguments increases. So do decorators outside a function. What's more, I haven't seen anybody posting any counterexamples to show that it doesn't suck for common use cases. Indeed, at the moment I don't even recall seeing any examples of class decorators being used without arguments! Well, here's how my use case would look if I had class decorators: @IOClass class MyClass: ... Does that count? My decorator wouldn't need any arguments, because it looks inside the class for all the information it needs. [1] That's actually a general solution to Phillip's concern: the decorator can always look for attributes in the class (or the class's __dict__ if you don't want them inherited) for large amounts of information that wouldn't comfortably fit up the top. That's an extra degree of freedom that we don't have with functions. - - - - - [1] Actually I would probably give it one optional argument, the name to register under if different from the class name. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | Carpe post meridiam! | Christchurch, New Zealand | (I'm not a morning person.) | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
[promted by Phillip Eby's post, but not in response so content snipped] I think we both want class decorators as a more fine grained substitute for __metaclass__ (fine grained as in declared per-class-instance instead of this-class-and-all-its-children). I can think of three ways class decorators are used: 1) register pattern, use a class attribute or two to stick the class in a lookup table and optionally delete the meta-attribs from the class 2) __init__ pattern, examine the class and maybe munge it based on attribs 3) __new__ pattern, consider the class a namespace and use the attribs to generate an essentially new class based on the attribs (the main difference between 2 and 3 is that the __new__ caseis a total tear down and rebuild where 2 tends towards tweaks on the parent) Classes have a unique property in that they are the easiest way to make little namespaces in python. Modules require lots of file clutter, functions as namespaces require digging into the internals, dicts as namespaces require more finger typing and don't support attribute access. It turns out I have two use cases for class decorators and didn't even know it. One is the 'register' pattern that started this thread. In that case I just want to move the metadata outside the class (the @register(db_id=20) case) and the rest of the class definition is honest to goodness overriding of a method or two from the parent class to change the behavior of its instances. The other pattern I hadn't thought of would be a '@namespace' decorator. A @namespace decorator would strip the attribs of all class-like qualities - it would strip the class of all descriptor magic (using descriptors, of course!). I just want a convenient bag to stick related items in. The first time I ever used metaclasses was to make PLY[1] (a SLR parser) use per-class namespaces for lexers and grammars instead of per-module. The metaclass chewed through all the class attributes and returned a class that shared no attributes with the original - each class was basically a config file. The decorator version would be spelled '@make_a_lexer_class' or '@make_a_grammar_class'. PEAK and Zope seem like they do a mix of __init__ and __new__, my current use cases are just 'notice' (I'm not a user, so feel free to correct). I like the func-like decorator syntax because I have just a bit of metadata that I'd like to move outside the class. PEAK/Zope sounds like they use classes as a mix of class and namespace. Their class decorator would return a hybrid class that has applied the namespace parts (post processing) to the class parts. A partly new class. I'd like that spelled: @tweak_this_class class MyClass: namespacey_stuff = ... def class_thing(self, foo): pass That leaves a class decorator behaving like a function decorator. It is a declaration that the final product (maybe a completely different thing as in the PLY case) is the result of the tweak_this_class function. Writing the above certainly helped me clear up my thoughts on the issue, I hope it hasn't gone and muddied it for anyone else :) -jackdied [1] http://savannah.nongnu.org/projects/ply/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 03:09 PM 3/30/2006 +1200, Greg Ewing wrote: Well, here's how my use case would look if I had class decorators: @IOClass class MyClass: ... Does that count? My decorator wouldn't need any arguments, because it looks inside the class for all the information it needs. [1] That's actually a general solution to Phillip's concern: the decorator can always look for attributes in the class (or the class's __dict__ if you don't want them inherited) for large amounts of information that wouldn't comfortably fit up the top. That's an extra degree of freedom that we don't have with functions. I fail to see how this is an improvement. It sounds like the worst of all possible worlds to me -- *much* worse than the status quo. I use class decorators to declare metadata *about* attributes, so having to add a bunch of attributes for that *and* having to stick a funny hat on top of the class is definitely not an improvement for me. Are you actually *using* this IOClass thing, or is this just a hypothetical proposal? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 11:09 PM 3/29/2006 -0500, Jack Diederich wrote: I think we both want class decorators as a more fine grained substitute for __metaclass__ (fine grained as in declared per-class-instance instead of this-class-and-all-its-children). I can think of three ways class decorators are used: 1) register pattern, use a class attribute or two to stick the class in a lookup table and optionally delete the meta-attribs from the class 2) __init__ pattern, examine the class and maybe munge it based on attribs 3) __new__ pattern, consider the class a namespace and use the attribs to generate an essentially new class based on the attribs My uses of class decorators don't fit any of your descriptions -- neither do Zope's for that matter, at least not the ones I'm familiar with. implements(IFoo) doesn't use any class attributes or delete meta attributes from the class. The whole point of using these decorators is to *not* interfere with the class by putting metadata in attributes. This is especially true for PEAK's class decorators, which are used to do things like make security declarations about attributes offered by the class (see the examples I posted previously in response to Fred's @class proposal). It turns out I have two use cases for class decorators and didn't even know it. One is the 'register' pattern that started this thread. In that case I just want to move the metadata outside the class (the @register(db_id=20) case) and the rest of the class definition is honest to goodness overriding of a method or two from the parent class to change the behavior of its instances. The other pattern I hadn't thought of would be a '@namespace' decorator. A @namespace decorator would strip the attribs of all class-like qualities - it would strip the class of all descriptor magic (using descriptors, of course!). I just want a convenient bag to stick related items in. This seems to me like a perfectly good place to use a metaclass. The place where metaclasses tend to quickly meet their downfall is in trying to support inheritance, especially multiple inheritance. If you just have a bag of data, however, inheritance doesn't enter into it and a metaclass is *perfect*. It doesn't even have to inherit from 'type', it can even be a classic class or a function, just as long as it takes a name, bases tuple, and dictionary. PEAK and Zope seem like they do a mix of __init__ and __new__, my current use cases are just 'notice' (I'm not a user, so feel free to correct). I like the func-like decorator syntax because I have just a bit of metadata that I'd like to move outside the class. PEAK/Zope sounds like they use classes as a mix of class and namespace. Their class decorator would return a hybrid class that has applied the namespace parts (post processing) to the class parts. A partly new class. I'd like that spelled: @tweak_this_class class MyClass: namespacey_stuff = ... def class_thing(self, foo): pass You can do this quite easily now. Try running this code and see what happens: class Tweaker(object): def __class__(self, name, bases, cdict): print name, bases, cdict # put code here that returns something using name,bases,cdict return I'm a string! Tweak = Tweaker() class MyClass(Tweak): namespacey_stuff = foo def class_thing(self, foo): pass assert MyClass == I'm a string! If what you're making isn't a class, metaclasses work just fine. The return value of the __class__ method shown here will be bound to the name 'MyClass' when this runs. That leaves a class decorator behaving like a function decorator. They always did, and as far as I know, nobody has proposed they behave otherwise. It is a declaration that the final product (maybe a completely different thing as in the PLY case) is the result of the tweak_this_class function. Again, if you don't want a class to be the result, you can do this sort of thing quite easily now. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Mike Krell wrote: Greg Ewing greg.ewing at canterbury.ac.nz writes: I've just been playing around with metaclasses, and I think I've stumbled across a reason for having class decorators as an alternative to metaclasses for some purposes. There has also been discussion on the IronPython mailing list that class decorators would be a very useful syntax for expressing .NET attributes. http://lists.ironpython.com/pipermail/users-ironpython.com/2006-March/002007.html the same is true for Jython and Java annotations. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On 3/28/06, Guido van Rossum [EMAIL PROTECTED] wrote: I propose that someone start writing a Py3k PEP for class decorators. I don't think it's fair to the 2.5 release team to want to push this into 2.5 though; how about 2.6? Wasn't there already a (pretty small) patch? I guess it would be different now with the AST though. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On Tue, Mar 28, 2006 at 10:16:01AM -0800, Neal Norwitz wrote: On 3/28/06, Guido van Rossum [EMAIL PROTECTED] wrote: I propose that someone start writing a Py3k PEP for class decorators. I don't think it's fair to the 2.5 release team to want to push this into 2.5 though; how about 2.6? Wasn't there already a (pretty small) patch? I guess it would be different now with the AST though. I submitted one a couple years ago. The AST makes it obsolete though. I'd be happy to make a new AST friendly one (time to learn the AST..) -Jack ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 10:01 AM 3/28/2006 -0800, Guido van Rossum wrote: OK, I'm convinced (mostly by the awful hackery that Phillip so proudly exposed :-). Just as a historical note, here's where you previously rejected the same hackery as an argument for supporting class decorators: http://mail.python.org/pipermail/python-dev/2004-March/043462.html Ironically, the subsequent discussion following the above message brought me around to your point of view. :) Or more precisely, the subsequent discussion and examples convinced me that putting class decorators on top of the class was bad for readability, vs. putting them in the body just after the docstring. As you said, the use cases are certainly very *different* than those for function/method decorators. So at this point I'd rather see a way to make my hackery go away (or become part of the standard library in some fashion) rather than simply mimic @decorators for classes. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On 3/28/06, Phillip J. Eby [EMAIL PROTECTED] wrote: At 10:01 AM 3/28/2006 -0800, Guido van Rossum wrote: OK, I'm convinced (mostly by the awful hackery that Phillip so proudly exposed :-). Just as a historical note, here's where you previously rejected the same hackery as an argument for supporting class decorators: http://mail.python.org/pipermail/python-dev/2004-March/043462.html Ironically, the subsequent discussion following the above message brought me around to your point of view. :) Or more precisely, the subsequent discussion and examples convinced me that putting class decorators on top of the class was bad for readability, vs. putting them in the body just after the docstring. As you said, the use cases are certainly very *different* than those for function/method decorators. So at this point I'd rather see a way to make my hackery go away (or become part of the standard library in some fashion) rather than simply mimic @decorators for classes. That's fine. But there's also the C#/Java POV. I'm somehow concerned that any mechanism that puts the syntax inside the class body is somehow going to have a hackish implementation, but I'd be glad to be proven wrong, so please come up with a concrete proposal! -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 11:04 AM 3/28/2006 -0800, Guido van Rossum wrote:
That's fine. But there's also the C#/Java POV.
Can someone point me to examples of C# class attributes and Java
annotations that they'd like to use with this mechanism? I would indeed
like to see how those use cases compare with mine.
I'm somehow concerned
that any mechanism that puts the syntax inside the class body is
somehow going to have a hackish implementation,
That depends on whether there's syntax support, or it's just something that
happens at runtime. I'm not opposed to having syntax, I just don't think
that putting the syntax outside the class is the best thing for most of my
class decorator use cases.
Nonetheless, neither special syntax nor hackiness is really required. For
example, your proposal here:
http://mail.python.org/pipermail/python-dev/2004-April/043913.html
was to use a '__decorators__' attribute, which would then be processed by a
metaclass. However, this could simply become a standard magic attribute
ala __metaclass__, and be processed by the class creation machinery after
first invoking the correct metaclass.
I don't really *like* that approach, but it requires neither syntax nor
hacks, (nor a custom metaclass) and it could be syntax-sugared later.
For that matter, if this __decorators__ machinery existed, the existing
advisor implementation could become a lot less hacky. The only magical bit
at that point would be doing something like:
locals = sys._getframe(n).f_locals.
locals.setdefault('__decorators__',[]).append(decorator)
And this of course could go in a function in the stdlib, rather than have
everybody writing their own _getframe() hacks.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Phillip J. Eby wrote: http://mail.python.org/pipermail/python-dev/2004-March/043462.html Or more precisely, the subsequent discussion and examples convinced me that putting class decorators on top of the class was bad for readability, vs. putting them in the body just after the docstring. I just looked at that discussion, and I didn't see any argument to refute the thesis that if decorators-on-top is good enough for functions, it should be good enough for classes. In fact it seems to have been written before the on-top decorator syntax existed, so there's no discussion of the issue at all. Seems to me that, since we now already have @-decorators for functions, a *very* good reason will be needed for requiring a completely different syntax to get exactly the same effect for classes. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 02:55 PM 3/29/2006 +1200, Greg Ewing wrote: Phillip J. Eby wrote: http://mail.python.org/pipermail/python-dev/2004-March/043462.html Or more precisely, the subsequent discussion and examples convinced me that putting class decorators on top of the class was bad for readability, vs. putting them in the body just after the docstring. I just looked at that discussion, and I didn't see any argument to refute the thesis that if decorators-on-top is good enough for functions, it should be good enough for classes. In fact it seems to have been written before the on-top decorator syntax existed, so there's no discussion of the issue at all. Seems to me that, since we now already have @-decorators for functions, a *very* good reason will be needed for requiring a completely different syntax to get exactly the same effect for classes. And here it is: because the use cases for class decorators are different. I routinely use them with things that take numerous keyword arguments, but this isn't nearly as common of a scenario for function decorators. Also, class decorators are far more likely to be just registering the class with something -- which means they don't deserve so prominent a location as to obscure the class itself. ObDisclaimer: this is my personal experience and opinion. Others may have different use cases in mind. I'm just pointing out that if @decorator support were added for classes, I wouldn't use it, because it's not actually an improvement over what I'm doing now. Function decorators, OTOH, *are* an improvement over what I did before, so I use them. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On Tuesday 28 March 2006 22:06, Phillip J. Eby wrote: And here it is: because the use cases for class decorators are different. This is vague. I routinely use them with things that take numerous keyword arguments, but this isn't nearly as common of a scenario for function decorators. The zope.formlib decorators are commonly used with many arguments; those construct the replacements and register with a class-specific registry. Also, class decorators are far more likely to be just registering the class with something -- which means they don't deserve so prominent a location as to obscure the class itself. I've not looked at the Java and C# examples, so I don't claim anything about those examples. For all the cases where I'm registering classes, however, it's not a local registry, but something that lives elsewhere in the system; it doesn't affect the class itself at all. I'm happy for that use case to be supported in other ways than prefixing the class with decorator-syntax stuff. ObDisclaimer: this is my personal experience and opinion. Others may have different use cases in mind. I'm just pointing out that if @decorator support were added for classes, I wouldn't use it, because it's not actually an improvement over what I'm doing now. So it doesn't apply. I suspect this is something for which familiarity with the use cases for the Java and C# precedents would help. For Zope 3, we have decorators that work with the component architecture (I'm sure Phillip is familiar with these). They're used with functions to indicate that the function adapts a particular kind of object, or that it implements or provides a particular interface. We have different functions that get used for this purpose in classes that are executed within the body of the class. There's some merit to being able to use a single set of functions in both cases, since the use cases are the same. I'm not sure I'd want to change the existing pattern, though, since it's already so widespread within the Zope 3 codebase (including 3rd-party components). -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 11:35 PM 3/28/2006 -0500, Fred L. Drake, Jr. wrote: For Zope 3, we have decorators that work with the component architecture (I'm sure Phillip is familiar with these). They're used with functions to indicate that the function adapts a particular kind of object, or that it implements or provides a particular interface. We have different functions that get used for this purpose in classes that are executed within the body of the class. There's some merit to being able to use a single set of functions in both cases, since the use cases are the same. I'm not sure I'd want to change the existing pattern, though, since it's already so widespread within the Zope 3 codebase (including 3rd-party components). If we're using Zope 3 as an example, I personally find that: class Foo: Docstring here, blah blah blah implements(IFoo) is easier to read than: @implements(IFoo) class Foo: Docstring here, blah blah blah And it only gets worse when you have lots of arguments or multiple decorators. For some reason, this doesn't bother me with functions. But then, I can't remember how often I've actually needed to use two decorators on the same function, or how many times a function decorator's arguments took multiple lines to list. Both of these are routine occurrences for my class use cases. It's too bad this syntax is ambiguous: class Foo: Docstring here, blah blah blah @implements(IFoo) As this achieves a desirable highlighting of the specialness, without forcing the decorator outside the class. Oh well. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On Wednesday 29 March 2006 00:01, Phillip J. Eby wrote: If we're using Zope 3 as an example, I personally find that: class Foo: Docstring here, blah blah blah implements(IFoo) is easier to read than: I think the existing usage for classes is perfectly readable. The @-syntax works well for functions as well. For some reason, this doesn't bother me with functions. But then, I can't remember how often I've actually needed to use two decorators on the same function, or how many times a function decorator's arguments took multiple lines to list. For zope.formlib actions, I find there's usually only one decorator. Sometimes it fits comfortably on one line, and sometimes it takes two or three. For component architecture decorators, we find we commonly use two (zope.interface.implementer and zope.component.adapter) in tandem. This can be fairly verbose with multi-object adaptation, or really long package names. It's too bad this syntax is ambiguous: class Foo: Docstring here, blah blah blah @implements(IFoo) As this achieves a desirable highlighting of the specialness, without forcing the decorator outside the class. Oh well. Agreed, but... guess we can't have everything. On the other hand, something like: class Foo: Documentation is good. @class implements(IFoo) is not ambiguous. Hmm. It even says what it means. :-) -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On Wednesday 29 March 2006 00:48, Fred L. Drake, Jr. wrote: I think the existing usage for classes is perfectly readable. The @-syntax works well for functions as well. On re-reading what I wrote, I don't think I actually clarified the point I was trying to make originally. My point wasn't that I desparately need @-syntax for class decorators (I don't), or see it as inherantly superior in some way. It's much simpler than that: I just want to be able to use the same syntax for a group of use cases regardless of whether the target is a function or a class. This fits into the nice-to-have category for me, since the use case can be the same regardless of whether I'm decorating a class or a function. (I will note that when this use case applies to a function, it's usually a module-level function I'm decorating rather than a method.) My other example, the zope.formlib example, has only ever involved a single decorator, and is always a method. It could conceivably be applied to a nested class without much of a stretch, however. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Greg Ewing greg.ewing at canterbury.ac.nz writes: I've just been playing around with metaclasses, and I think I've stumbled across a reason for having class decorators as an alternative to metaclasses for some purposes. There has also been discussion on the IronPython mailing list that class decorators would be a very useful syntax for expressing .NET attributes. http://lists.ironpython.com/pipermail/users-ironpython.com/2006-March/002007.html Mike ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 07:20 PM 3/27/2006 +, Mike Krell wrote: Greg Ewing greg.ewing at canterbury.ac.nz writes: I've just been playing around with metaclasses, and I think I've stumbled across a reason for having class decorators as an alternative to metaclasses for some purposes. There has also been discussion on the IronPython mailing list that class decorators would be a very useful syntax for expressing .NET attributes. http://lists.ironpython.com/pipermail/users-ironpython.com/2006-March/002007.html PyProtocols and the zope.interface package both support inline class decorators called class advisors. They don't require any special syntax, and aren't much more complex than regular decorators. By defining an advisor like this: from protocols.advice import addClassAdvisor def some_advisor(whatever_args): def callback(cls): print I can modify,cls,or replace it return cls addClassAdvisor(callback) you can then use it in a class body like so: class SomeClass: some_advisor(something) And the return value from 'callback' will replace SomeClass, just like a decorator replaces the function it's called on. The implementation should work with any Python version from 2.2 up. I'm not sure if it would work with IronPython. But if it doesn't, that would be a good indication of feature(s) that IronPython is missing. ;) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
On 3/27/06, Phillip J. Eby [EMAIL PROTECTED] wrote: PyProtocols and the zope.interface package both support inline class decorators called class advisors. They don't require any special syntax, and aren't much more complex than regular decorators. By defining an advisor like this: from protocols.advice import addClassAdvisor def some_advisor(whatever_args): def callback(cls): print I can modify,cls,or replace it return cls addClassAdvisor(callback) you can then use it in a class body like so: class SomeClass: some_advisor(something) And the return value from 'callback' will replace SomeClass, just like a decorator replaces the function it's called on. The implementation should work with any Python version from 2.2 up. I'm not sure if it would work with IronPython. But if it doesn't, that would be a good indication of feature(s) that IronPython is missing. ;) Just curious (and lazy): what magic is the implementation using that makes this work without a custom metaclass? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
At 08:02 PM 3/27/2006 -0800, Guido van Rossum wrote: Just curious (and lazy): what magic is the implementation using that makes this work without a custom metaclass? It registers a function as the __metaclass__ by poking it into the f_locals of the frame that's defining the class. This function has two attributes that denote 1) the callback to be invoked, and 2) the previous metaclass, forming a linked list of callbacks. When the class is created, this dummy metaclass function invokes the previous metaclass (or metaclass function) in the chain, then calls its callback on the result and returns that value, thus recursively unwinding the chain of advisors in the right order. (i.e., the same order as the advisors were called in the class body). The only part of the code that's even mildly complex is simulating Python's builtin rules for determining the default metaclass, so that when the recursion bottoms out you end up with an actual class with the right metaclass. And that's not so much complex as tedious. Anyway, the mechanism is orthogonal to the chosen metaclass, and thus can be used with classic classes or even ExtensionClasses if you're sufficiently masochistic. :) And AFAIK it's been in use by Zope 3 for some years now; in fact I think you were still working at Zope Corp. when Steve Alexander came up with the basic idea and I made the first practical implementation. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators
Phillip J. Eby wrote: It registers a function as the __metaclass__ by poking it into the f_locals of the frame that's defining the class. That is stunningly brilliant! I'd nominate it for Hack of the Year if there were such an award. It's far too magical for me to feel like actually using it, though. Class decorators would be so much more straightforward and explicit. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Class decorators
I've just been playing around with metaclasses, and I think I've stumbled across a reason for having class decorators as an alternative to metaclasses for some purposes. The metaclass I wrote was for the purpose of adding a class to a registry, the reason for which isn't important here. It worked, but I was surprised to find that it not only registered the classes that I made it the metaclass of, but all subclasses of those classes as well. I'm not sure whether that's really the behaviour I want, and I can imagine some cases in which it's definitely not what I'd want. The general principle brought out here is that when you use a metaclass, it gets inherited by subclasses, but if we had class decorators, they would only affect to the classes that you explicitly applied them to. I think there are uses for both behaviours. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators vs metaclasses
On 11/5/05, Alex Martelli [EMAIL PROTECTED] wrote:
On 11/4/05, Eyal Lotem [EMAIL PROTECTED] wrote:
I have a few claims, some unrelated, and some built on top of each
other. I would like to hear your responses as to which are
convincing, which arne't, and why. I think that if these claims are
true, Python 3000 should change quite a bit.
A. Metaclass code is black magic and few understand how it works,
while decorator code is mostly understandable, even by non-gurus.
I disagree. I've held many presentations and classes on both
subjects, and while people may INITIALLY feel like metaclasses are
black magic, as soon as I've explained it the fear dissipates. It all
boils down do understanding that:
class Name(Ba,Ses): body
means
Name = suitable_metaclass('Name', (Ba,Ses), dict-built-by-body)
which isn't any harder than understanding that
@foo(bar)
def baz(args): ...
means
def baz(args): ...
baz = foo(bar)(baz)
I disagree again. My experience is that metaclass code is very hard to
understand. Especially when it starts doing non-trivial things, such
as using a base metaclass class that is parametrized by metaclass
attributes in its subclasses. Lookups of attributes in the base
metaclass methods is mind boggling (is it searching them in the base
metaclass, the subclass, the instance [which is the class]?). The
same code would be much easier to understand with class decorators.
B. One of Decorators' most powerful features is that they can
mixed-and-matched, which makes them very powerful for many purposes,
while metaclasses are exclusive, and only one can be used. This is
Wrong. You can mix as many metaclasses as you wish, as long as
they're properly coded for multiple inheritance (using super, etc) --
just inherit from them all. This is reasonably easy to automate (see
the last recipe in the 2nd ed of the Python Cookbook), too.
Multiple inheritence is an awful way to mix class fucntionalities
though. Lets take a simpler example. Most UT frameworks use a
TestCase base class they inherit from to implement setup, tearDown,
and then inherit from it again to implement the test itself. I argue
this is a weak approach, because then mixing/matching setups is
difficult. You would argue this is not the case, because of the
ability to multiply-inherit from test cases, but how easy is the
equivalent of:
@with_setup('blah')
@with_other_setup('bleh')
def my_test():
# the blah setup and bleh other setup are up and usable here,
# and will be torn down at the end of this test
The equivalent of this requires a lot more work and violating DRY.
Creating a specific function to multiply inherit from TestCases is a
possible solution, but it is much more conceptually complex, and needs
to be reimplemented in the next scenario (Metaclasses for example).
especially problematic as some classes may assume their subclasses
must use their respective metaclasses. This means classdecorators are
strictly more powerful than metaclasses, without cumbersome
convertions between metaclass mechanisms and decorator mechanisms.
The assertion that classdecorators are strictly more powerful than
custom metaclasses is simply false. How would you design
classdecorator XXX so that
@XXX
class Foo: ...
allows 'print Foo' to emit 'this is beautiful class Foo', for example?
the str(Foo) implicit in print calls type(Foo).__str__(Foo), so you
do need a custom type(Foo) -- which is all that is meant by a custom
metaclass... a custom type whose instances are classes, that's all.
I would argue that this is not such a useful feature, as in that case
you can simply use a factory object instead of a class. If this
feature remains, that's fine, but the fact it allows for a weak form
of decoration of classes should not kill the concept of class
decorators.
The only reason of using metaclasses rather than factory objects, in
my experience, was that references to class objects are considered
different than references to factories (by pickle and deepcopy, and
maybe others) and that can be a useful feature. This feature can be
implemented in more readable means though.
C. Interesting uses of classdecorators are allowing super-calling
without redundantly specifying the name of your class, or your
superclass.
Can you give an example?
@anotherclassdecorator
@supercallerclass
class MyClass(object):
@supercaller
def my_method(self, supcaller, x, y, z):
...
result = supcaller.my_method(x, y, z)
...
Could be nice to remove the need for decorating the class, and only
decorating the methods, but the method decorators get a function
object, not a method object, so its more difficult (perhaps portably
impossible?) to do this.
Note that __metaclass__ = superclasscaller could also work, but then
combining anotherclassdecorator would require a lot more code at
worst, or a complex mechanism to combine metaclasses via multiple
inheritence at best.
[Python-Dev] Class decorators vs metaclasses
I have a few claims, some unrelated, and some built on top of each other. I would like to hear your responses as to which are convincing, which arne't, and why. I think that if these claims are true, Python 3000 should change quite a bit. A. Metaclass code is black magic and few understand how it works, while decorator code is mostly understandable, even by non-gurus. B. One of Decorators' most powerful features is that they can mixed-and-matched, which makes them very powerful for many purposes, while metaclasses are exclusive, and only one can be used. This is especially problematic as some classes may assume their subclasses must use their respective metaclasses. This means classdecorators are strictly more powerful than metaclasses, without cumbersome convertions between metaclass mechanisms and decorator mechanisms. C. Interesting uses of classdecorators are allowing super-calling without redundantly specifying the name of your class, or your superclass. D. Python seems to be incrementally adding power to the core language with these features, which is great, but it also causes significant overlapping of language features, which I believe is something to avoid when possible. If metaclasses are replaced with class decorators, then suddenly inheritence becomes a redundant feature. E. If inheritence is a redundant feature, it can be removed and an inherit class decorator can be used. This could also reduce all the __mro__ clutter from the language along with other complexities, into alternate implementations of the inherit classdecorator. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class decorators vs metaclasses
On 11/4/05, Eyal Lotem [EMAIL PROTECTED] wrote:
I have a few claims, some unrelated, and some built on top of each
other. I would like to hear your responses as to which are
convincing, which arne't, and why. I think that if these claims are
true, Python 3000 should change quite a bit.
A. Metaclass code is black magic and few understand how it works,
while decorator code is mostly understandable, even by non-gurus.
I disagree. I've held many presentations and classes on both
subjects, and while people may INITIALLY feel like metaclasses are
black magic, as soon as I've explained it the fear dissipates. It all
boils down do understanding that:
class Name(Ba,Ses): body
means
Name = suitable_metaclass('Name', (Ba,Ses), dict-built-by-body)
which isn't any harder than understanding that
@foo(bar)
def baz(args): ...
means
def baz(args): ...
baz = foo(bar)(baz)
B. One of Decorators' most powerful features is that they can
mixed-and-matched, which makes them very powerful for many purposes,
while metaclasses are exclusive, and only one can be used. This is
Wrong. You can mix as many metaclasses as you wish, as long as
they're properly coded for multiple inheritance (using super, etc) --
just inherit from them all. This is reasonably easy to automate (see
the last recipe in the 2nd ed of the Python Cookbook), too.
especially problematic as some classes may assume their subclasses
must use their respective metaclasses. This means classdecorators are
strictly more powerful than metaclasses, without cumbersome
convertions between metaclass mechanisms and decorator mechanisms.
The assertion that classdecorators are strictly more powerful than
custom metaclasses is simply false. How would you design
classdecorator XXX so that
@XXX
class Foo: ...
allows 'print Foo' to emit 'this is beautiful class Foo', for example?
the str(Foo) implicit in print calls type(Foo).__str__(Foo), so you
do need a custom type(Foo) -- which is all that is meant by a custom
metaclass... a custom type whose instances are classes, that's all.
C. Interesting uses of classdecorators are allowing super-calling
without redundantly specifying the name of your class, or your
superclass.
Can you give an example?
D. Python seems to be incrementally adding power to the core language
with these features, which is great, but it also causes significant
overlapping of language features, which I believe is something to
avoid when possible. If metaclasses are replaced with class
decorators, then suddenly inheritence becomes a redundant feature.
And how do you customize what print Foo emits, as above?
E. If inheritence is a redundant feature, it can be removed and an
inherit class decorator can be used. This could also reduce all the
__mro__ clutter from the language along with other complexities, into
alternate implementations of the inherit classdecorator.
How do you propose to get exactly the same effects as inheritance
(affect every attribute lookup on a class and its instances) without
inheritance? Essentially, inheritance is automated delegation
obtained by having getattr(foo, 'bar') look through a chain of objects
(essentially the __mro__) until an attribute named 'bar' is found in
one of those objects, plus a few minor but useful side effects, e.g.
on isinstance and issubclass, and the catching of exceptions in
try/except statements. How would any mechanism allowing all of these
uses NOT be inheritance?
Alex
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
