Re: [Python-Dev] anonymous blocks
Guido van Rossum wrote: Fredrik, what does your intuition tell you? having been busy with other stuff for nearly a week, and seeing that the PEP is now at version 1.22, my intuition tells me that it's time to read the PEP again before I have any opinion on anything ;-) /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
RE: [Python-Dev] Anonymous blocks: Thunks or iterators?
[Greg Ewing] Elegant as the idea behind PEP 340 is, I can't shake the feeling that it's an abuse of generators. It seems to go to a lot of trouble and complication so you can write a generator and pretend it's a function taking a block argument. [Guido] Maybe. You're not the first one saying this and I'm not saying no outright, but I'd like to defend the PEP. There are a number of separate ideas that all contribute to PEP 340. One is turning generators into more general coroutines: continue EXPR passes the expression to the iterator's next() method (renamed to __next__() to work around a compatibility issue and because it should have been called that in the first place), and in a generator this value can be received as the return value of yield. Incidentally this makes the generator *syntax* more similar to Ruby (even though Ruby uses thunks, and consequently uses return instead of continue to pass a value back). I'd like to have this even if I don't get the block statement. Completely agree. Maybe we should have PEP 340 push just that, and make a PEP 341 independently for resource-cleanup (which assumes 340)? [snip] The other problem with thunks is that once we think of them as the anonymous functions they are, we're pretty much forced to say that a return statement in a thunk returns from the thunk rather than from the containing function. Doing it any other way would cause major weirdness when the thunk were to survive its containing function as a closure (perhaps continuations would help, but I'm not about to go there :-). But then an IMO important use case for the resource cleanup template pattern is lost. I routinely write code like this: def findSomething(self, key, default=None): self.lock.acquire() try: for item in self.elements: if item.matches(key): return item return default finally: self.lock.release() and I'd be bummed if I couldn't write this as def findSomething(self, key, default=None): block synchronized(self.lock): for item in self.elements: if item.matches(key): return item return default Okay, you've convinced me. The only way I can think of to get the effect I've been wanting would be to recompile the template function every time that it's executed with a different block. Call it a Python _re_processor ;). Although you could memoize the the resultant bytecode, etc., it would still be pretty slow, and you wouldn't be able to alter (rebind) the thunk once you'd entered the caller. Even then, you'd have the cell issues you mentioned, trying to push values from the thunk's original scope. Bah. It's so tempting on the semantic level, but the implementation's a bear. Robert Brewer MIS Amor Ministries [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Guido van Rossum [EMAIL PROTECTED] writes: [Greg Ewing] Elegant as the idea behind PEP 340 is, I can't shake the feeling that it's an abuse of generators. It seems to go to a lot of trouble and complication so you can write a generator and pretend it's a function taking a block argument. Maybe. You're not the first one saying this and I'm not saying no outright, but I'd like to defend the PEP. This is kind of my point too; I'm not saying that I really prefer the thunk solution, just that I want to see it mentioned. I think the making-generators-more-sexy thing is nice, but I'm think that's almost orthogonal. [...] Even without a block-statement, these two changes make yield look a lot like invoking a thunk -- but it's more efficient, since calling yield doesn't create a frame. The main advantage of thunks that I can see is that you can save the thunk for later, I also find them somewhat easier to understand. like a callback for a button widget (the thunk then becomes a closure). You can't use a yield-based block for that (except in Ruby, which uses yield syntax with a thunk-based implementation). But I have to say that I almost see this as an advantage: I think I'd be slightly uncomfortable seeing a block and not knowing whether it will be executed in the normal control flow or later. Defining an explicit nested function for that purpose doesn't have this problem for me, because I already know that the 'def' keyword means its body is executed later. The other problem with thunks is that once we think of them as the anonymous functions they are, we're pretty much forced to say that a return statement in a thunk returns from the thunk rather than from the containing function. Doing it any other way would cause major weirdness when the thunk were to survive its containing function as a closure (perhaps continuations would help, but I'm not about to go there :-). I'm not so sure about this. Did you read this mail: http://mail.python.org/pipermail/python-dev/2005-April/052970.html ? In this proposal, you have to go to some effort to make the thunk survive the block, and I think if weirdness results, that's the programmer's problem. But then an IMO important use case for the resource cleanup template pattern is lost. I routinely write code like this: def findSomething(self, key, default=None): self.lock.acquire() try: for item in self.elements: if item.matches(key): return item return default finally: self.lock.release() and I'd be bummed if I couldn't write this as def findSomething(self, key, default=None): block synchronized(self.lock): for item in self.elements: if item.matches(key): return item return default If you can't write it this way, the thunk proposal is dead. I'd like to reconsider a thunk implementation. It would be a lot simpler, doing just what is required without any jiggery pokery with exceptions and break/continue/return statements. It would be easy to explain what it does and why it's useful. I don't know. In order to obtain the required local variable sharing between the thunk and the containing function I believe that every local variable used or set in the thunk would have to become a 'cell' (our mechanism for sharing variables between nested scopes). Yes. Cells slow down access somewhat compared to regular local variables. So make them faster. I'm not sure I think this is a good argument. You could also do some analysis and treat variables that are only accessed or written in the block as normal locals. This all makes a block-created thunk somewhat different from an anonymous function, to be sure. But the creating syntax is different, so I don't know if I care (hell, the invoking syntax could be made different too, but I really don't think that's a good idea). Perhaps not entirely coincidentally, the last example above (findSomething() rewritten to avoid a return inside the block) shows that, unlike for regular nested functions, we'll want variables *assigned to* by the thunk also to be shared with the containing function, even if they are not assigned to outside the thunk. I swear I didn't create the example for this purpose -- it just happened. Oh, absolutely. On the other hand, a thunk implementation has the potential to easily handle multiple block arguments, if a suitable syntax could ever be devised. It's hard to see how that could be done in a general way with the generator implementation. Right, but the use cases for multiple blocks seem elusive. If you really want to have multiple blocks with yield, I suppose we could use yield/n to yield to the n'th block argument, or perhaps yieldn. :-) Hmm, it's nearly *May* 1... :) Cheers, mwh -- I'm a keen cyclist and I stop at red lights. Those who don't
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Brian Sabbey [EMAIL PROTECTED] writes: It is possible to implement thunks without them creating their own frame. They can reuse the frame of the surrounding function. So a new frame does not need to be created when the thunk is called, and, much like with a yield statement, the frame is not taken down when the thunk completes running. The implementation just needs to take care to save and restore members of the frame that get clobbered when the thunk is running. Woo. That's cute. Cheers, mwh -- SCSI is not magic. There are fundamental technical reasons why it is necessary to sacrifice a young goat to your SCSI chain now and then. -- John Woods ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
On Thu, Apr 28, 2005, Brian Sabbey wrote: It is possible to implement thunks without them creating their own frame. They can reuse the frame of the surrounding function. So a new frame does not need to be created when the thunk is called, and, much like with a yield statement, the frame is not taken down when the thunk completes running. The implementation just needs to take care to save and restore members of the frame that get clobbered when the thunk is running. Cells would of course not be required if the thunk does not create its own frame. Maybe. It's not clear whether your thunks are lexical (I haven't been following the discussion closely). If it's not lexical, how do locals get handled without cells? -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ It's 106 miles to Chicago. We have a full tank of gas, a half-pack of cigarettes, it's dark, and we're wearing sunglasses. Hit it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
[Michael Hudson] I think the making-generators-more-sexy thing is nice, but I'm think that's almost orthogonal. Not entirely. I agree that continue EXPR calling next(EXPR) which enables yield-expressions is entirely orthogonal. But there are already two PEPs asking for passing exceptions and/or cleanup into generators and from there it's only a small step to using them as resource allocation/release templates. The small step part is important -- given that we're going to do that work on generators anyway, I expect the changes to the compiler and VM to support the block statement are actually *less* than the changes needed to support thunks. No language feature is designed in isolation. Did you read this mail: http://mail.python.org/pipermail/python-dev/2005-April/052970.html ? In this proposal, you have to go to some effort to make the thunk survive the block, and I think if weirdness results, that's the programmer's problem. It's not a complete proposal though. You say And grudgingly, I guess you'd need to make returns behave like that anyway (meaning they should return from the containing function). But you don't give a hint on how that could be made to happen, and I expect that by the time you've figured out a mechanism, thunks aren't all that simple any more. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Jim Jewett wrote: The only members that need special attention are (f_code, f_lasti) and possibly (f_blockstack, f_iblock). You don't even need to take care of f_code. The thunk and its surrounding function can share the same code. The thunk gets compiled into the function the same way the body of a for loop would. (f_code, f_lasti) would need to be replaced with a stack of pairs. Finishing a code string would mean popping this stack, rather than popping the whole frame. There doesn't need to be a stack; each thunk can store its own f_lasti. One also needs to store f_back, and, to avoid exception weirdness, f_exc_XXX. In this way, calling the thunk is much like resuming a generator. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
On 4/29/05, Brian Sabbey [EMAIL PROTECTED] wrote: Jim Jewett wrote: The only members that need special attention are (f_code, f_lasti) and possibly (f_blockstack, f_iblock). You don't even need to take care of f_code. The thunk and its surrounding function can share the same code. The thunk gets compiled into the function the same way the body of a for loop would. This only works if you already know what the thunk's code will be when you compile the function. (Just splicing it in messes up jump targets.) One also needs to store f_back, and, to avoid exception weirdness, f_exc_XXX. f_back lists the previous stack frame (which shouldn't change during a thunk[1]), and f_exc_XXX is for the most recent exception -- I don't see any reason to treat thunks differently from loop bodies in that regard. [1] If the thunk calls another function (that needs its own frame), then that is handled the same as any regular function call. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
On 29 apr 2005, at 20.10, Brian Sabbey wrote: [...] The thunk and its surrounding function can share the same code. The thunk gets compiled into the function the same way the body of a for loop would. This seems really, truly, nasty! Wouldn't this require you to check the source code of the function you want to integrate your thunk into to avoid namespace collisions? Well, no, not to avoid collisions I guess, if it's truly regarded as part of the function. But this means it would use the function's global namespace, etc. You'd be unable to use anything from the scopes in which the thunk is defined, which makes it really, really ... wierd. Or have I not gotten it? //Simon ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Greg Ewing [EMAIL PROTECTED] writes: Are there any objective reasons to prefer a generator implementation over a thunk implementation? I, too, would like to see an answer to this question. I'd like to see an answer in the PEP, too. Cheers, mwh -- All obscurity will buy you is time enough to contract venereal diseases. -- Tim Peters, python-dev ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
[Greg Ewing] Elegant as the idea behind PEP 340 is, I can't shake the feeling that it's an abuse of generators. It seems to go to a lot of trouble and complication so you can write a generator and pretend it's a function taking a block argument. Maybe. You're not the first one saying this and I'm not saying no outright, but I'd like to defend the PEP. There are a number of separate ideas that all contribute to PEP 340. One is turning generators into more general coroutines: continue EXPR passes the expression to the iterator's next() method (renamed to __next__() to work around a compatibility issue and because it should have been called that in the first place), and in a generator this value can be received as the return value of yield. Incidentally this makes the generator *syntax* more similar to Ruby (even though Ruby uses thunks, and consequently uses return instead of continue to pass a value back). I'd like to have this even if I don't get the block statement. The second is a solution for generator resource cleanup. There are already two PEPs proposing a solution (288 and 325) so I have to assume this addresses real pain! The only new twist offered by PEP 340 is a unification of the next() API and the resource cleanup API: neither PEP 288 nor PEP 325 seems to specify rigorously what should happen if the generator executes another yield in response to a throw() or close() call (or whether that should even be allowed); PEP 340 takes the stance that it *is* allowed and should return a value from whatever call sent the exception. This feels right, especially together with the previous feature: if yield can return a value as if it were a function call, it should also be allowed to raise an exception, and catch or propagate it with impunity. Even without a block-statement, these two changes make yield look a lot like invoking a thunk -- but it's more efficient, since calling yield doesn't create a frame. The main advantage of thunks that I can see is that you can save the thunk for later, like a callback for a button widget (the thunk then becomes a closure). You can't use a yield-based block for that (except in Ruby, which uses yield syntax with a thunk-based implementation). But I have to say that I almost see this as an advantage: I think I'd be slightly uncomfortable seeing a block and not knowing whether it will be executed in the normal control flow or later. Defining an explicit nested function for that purpose doesn't have this problem for me, because I already know that the 'def' keyword means its body is executed later. The other problem with thunks is that once we think of them as the anonymous functions they are, we're pretty much forced to say that a return statement in a thunk returns from the thunk rather than from the containing function. Doing it any other way would cause major weirdness when the thunk were to survive its containing function as a closure (perhaps continuations would help, but I'm not about to go there :-). But then an IMO important use case for the resource cleanup template pattern is lost. I routinely write code like this: def findSomething(self, key, default=None): self.lock.acquire() try: for item in self.elements: if item.matches(key): return item return default finally: self.lock.release() and I'd be bummed if I couldn't write this as def findSomething(self, key, default=None): block synchronized(self.lock): for item in self.elements: if item.matches(key): return item return default This particular example can be rewritten using a break: def findSomething(self, key, default=None): block synchronized(self.lock): for item in self.elements: if item.matches(key): break else: item = default return item but it looks forced and the transformation isn't always that easy; you'd be forced to rewrite your code in a single-return style which feels too restrictive. I'd like to reconsider a thunk implementation. It would be a lot simpler, doing just what is required without any jiggery pokery with exceptions and break/continue/return statements. It would be easy to explain what it does and why it's useful. I don't know. In order to obtain the required local variable sharing between the thunk and the containing function I believe that every local variable used or set in the thunk would have to become a 'cell' (our mechanism for sharing variables between nested scopes). Cells slow down access somewhat compared to regular local variables. Perhaps not entirely coincidentally, the last example above (findSomething() rewritten to avoid a return inside the block) shows that, unlike for regular nested functions, we'll want variables *assigned to* by the thunk also to be shared with the
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Guido van Rossum wrote: The main advantage of thunks that I can see is that you can save the thunk for later, like a callback for a button widget (the thunk then becomes a closure). Or pass it on to another function. This is something we haven't considered -- what if one resource-acquision- generator (RAG?) wants to delegate to another RAG? With normal generators, one can always use the pattern for x in sub_generator(some_args): yield x But that clearly isn't going to work if the generators involved are RAGs, because the exceptions passed in are going to be raised at the point of the yield in the outer RAG, and the inner RAG isn't going to get finalized (assuming the for-loop doesn't participate in the finalization protocol). To get the finalization right, the inner generator needs to be invoked as a RAG, too: block sub_generator(some_args): yield But PEP 340 doesn't say what happens when the block contains a yield. A thunk implementation wouldn't have any problem with this, since the thunk can be passed down any number of levels before being called, and any exceptions raised in it will be propagated back up through all of them. The other problem with thunks is that once we think of them as the anonymous functions they are, we're pretty much forced to say that a return statement in a thunk returns from the thunk rather than from the containing function. Urg, you're right. Unless return is turned into an exception in that case. And then I suppose break and return (and yield?) will have to follow suit. I'm just trying to think how Smalltalk handles this, since it must have a similar problem, but I can't remember the details. every local variable used or set in the thunk would have to become a 'cell' . Cells slow down access somewhat compared to regular local variables. True, but is the difference all that great? It's just one more C-level indirection, isn't it? we'll want variables *assigned to* by the thunk also to be shared with the containing function, Agreed. We'd need to add a STORE_CELL bytecode or something for this. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Guido van Rossum wrote: Even without a block-statement, these two changes make yield look a lot like invoking a thunk -- but it's more efficient, since calling yield doesn't create a frame. I like PEP 340 a lot, probably as much or more than any thunk ideas I've seen. But I want to defend thunks here a little. It is possible to implement thunks without them creating their own frame. They can reuse the frame of the surrounding function. So a new frame does not need to be created when the thunk is called, and, much like with a yield statement, the frame is not taken down when the thunk completes running. The implementation just needs to take care to save and restore members of the frame that get clobbered when the thunk is running. Cells would of course not be required if the thunk does not create its own frame. The main advantage of thunks that I can see is that you can save the thunk for later, like a callback for a button widget (the thunk then becomes a closure). You can't use a yield-based block for that (except in Ruby, which uses yield syntax with a thunk-based implementation). But I have to say that I almost see this as an advantage: I think I'd be slightly uncomfortable seeing a block and not knowing whether it will be executed in the normal control flow or later. Defining an explicit nested function for that purpose doesn't have this problem for me, because I already know that the 'def' keyword means its body is executed later. I would also be uncomfortable if the thunk could be called at a later time. This can be disallowed by raising an exception if such an attempt is made. Such a restriction would not be completely arbitrary. One consequence of having the thunk borrow its surrounding function's frame is that it does not make much sense, implementationally speaking, to allow the thunk to be called at a later time (although I do realize that it's difficult to implement is not a good argument for anything). The other problem with thunks is that once we think of them as the anonymous functions they are, we're pretty much forced to say that a return statement in a thunk returns from the thunk rather than from the containing function. Doing it any other way would cause major weirdness when the thunk were to survive its containing function as a closure (perhaps continuations would help, but I'm not about to go there :-). If it is accepted that the thunk won't be callable at a later time, then I think it would seem normal that a return statement would return from the surrounding function. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
(BTW, I'm trying to update the PEP with a discussion of thunks.) [Guido] The main advantage of thunks that I can see is that you can save the thunk for later, like a callback for a button widget (the thunk then becomes a closure). [Greg] Or pass it on to another function. This is something we haven't considered -- what if one resource-acquision- generator (RAG?) wants to delegate to another RAG? With normal generators, one can always use the pattern for x in sub_generator(some_args): yield x But that clearly isn't going to work if the generators involved are RAGs, because the exceptions passed in are going to be raised at the point of the yield in the outer RAG, and the inner RAG isn't going to get finalized (assuming the for-loop doesn't participate in the finalization protocol). To get the finalization right, the inner generator needs to be invoked as a RAG, too: block sub_generator(some_args): yield But PEP 340 doesn't say what happens when the block contains a yield. The same as when a for-loop contains a yield. The sub_generator is entirely unaware of this yield, since the local control flow doesn't actually leave the block (i.e., it's not like a break, continue or return statement). When the loop that was resumed by the yield calls next(), the block is resumed back after the yield. The generator finalization semantics guarantee (within the limitations of all finalization semantics) that the block will be resumed eventually. I'll add this to the PEP, too. I'd say that a yield in a thunk would be more troublesome: does it turn the thunk into a generator or the containing function? It would have to be the thunk, but then things get weird quickly (the caller of the thunk has to treat the result of the call as an iterator). A thunk implementation wouldn't have any problem with this, since the thunk can be passed down any number of levels before being called, and any exceptions raised in it will be propagated back up through all of them. The other problem with thunks is that once we think of them as the anonymous functions they are, we're pretty much forced to say that a return statement in a thunk returns from the thunk rather than from the containing function. Urg, you're right. Unless return is turned into an exception in that case. And then I suppose break and return (and yield?) will have to follow suit. But wasn't that exactly what you were trying to avoid? :-) I'm just trying to think how Smalltalk handles this, since it must have a similar problem, but I can't remember the details. every local variable used or set in the thunk would have to become a 'cell' . Cells slow down access somewhat compared to regular local variables. True, but is the difference all that great? It's just one more C-level indirection, isn't it? Alas not. It becomes a call to PyCell_Set() or PyCell_Get(). we'll want variables *assigned to* by the thunk also to be shared with the containing function, Agreed. We'd need to add a STORE_CELL bytecode or something for this. This actually exists -- it is used for when an outer function stores into a local that it shares with an inner function. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Skip Montanaro wrote: Guido or perhaps even (making for VAR optional in the for-loop syntax) Guido with Guido in synchronized(the_lock): Guido BODY This could be a new statement, so the problematic issue of implicit try/finally in every for statement wouldn't be necessary. That complication would only be needed for the above form. s/in/with/ to get PEP 310. A parallel which has been bugging me is the existence of the iterator protocol (__iter__, next()) which you can implement manually if you want, and the existence of generators, which provide a nice clean way of writing iterators as functions. I'm wondering if something similar can't be found for the __enter__/__exit__ resource protocol. Guido's recent screed crystallised the idea of writing resources as two-part generators: def my_resource(): print Hi! # Do entrance code yield None# Go on with the contents of the 'with' block print Bye! # Do exit code Giving the internal generator object an enter method that calls self.next() (expecting None to be returned), and an exit method that does the same (but expects StopIteration to be raised) should suffice to make this possible with a PEP 310 style syntax. Interestingly, with this approach, for dummy in my_resource() would still wrap the block of code in the entrance/exit code (because my_resource *is* a generator), but it wouldn't get the try/finally semantics. An alternative would be to replace the 'yield None' with a 'break' or 'continue', and create an object which supports the resource protocol and NOT the iterator protocol. Something like: def my_resource(): print Hi! # Do entrance code continue # Go on with the contents of the 'with' block print Bye! # Do exit code (This is currently a SyntaxError, so it isn't ambiguous in any way) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Nick Coghlan wrote: An alternative would be to replace the 'yield None' with a 'break' or 'continue', and create an object which supports the resource protocol and NOT the iterator protocol. Something like: def my_resource(): print Hi! # Do entrance code continue # Go on with the contents of the 'with' block print Bye! # Do exit code (This is currently a SyntaxError, so it isn't ambiguous in any way) That's a very interesting suggestion. I've been lurking, thinking about a way to use something like PEP 310 to help manage database transactions. Here is some typical code that changes something under transaction control: begin_transaction() try: changestuff() changemorestuff() except: abort_transaction() raise else: commit_transaction() There's a lot of boilerplate code there. Using your suggestion, I could write that something like this: def transaction(): begin_transaction() try: continue except: abort_transaction() raise else: commit_transaction() with transaction(): changestuff() changemorestuff() Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On 4/21/05, Guido van Rossum [EMAIL PROTECTED] wrote: for dummy in synchronized(the_lock): BODY or perhaps even (making for VAR optional in the for-loop syntax) with in synchronized(the_lock): BODY Then synchronized() could be written cleanly as follows: def synchronized(lock): lock.acquire() try: yield None finally: lock.release() How is this different from: def synchronized(lock): def synch_fn(block): lock.acquire() try: block() finally: lock.release() return synch_fn @synchronized def foo(): BLOCK True, it's non-obvious that foo is being immediately executed, but regardless I like the way synchronized is defined, and doesn't use yield (which in my opinion is a non-obvious solution) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Ka-Ping Yee wrote: Can you explain what you meant by currying here? I know what the word curry means, but i am having a hard time seeing how it applies to your example. It's currying in the sense that instead of one function which takes all the args at once, you have a function that takes some of them (all except the thunk) and returns another one that takes the rest (the thunk). Could you make up an example that uses more arguments? def with_file(filename, mode): def func(block): f = open(filename, mode) try: block(f) finally: f.close() return func Usage example: with_file(foo.txt, w) as f: f.write(My hovercraft is full of parrots.) Does that help? -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Brian Sabbey wrote:
I made an example implementation, and this wasn't an issue. It took
some code to stick the thunk into the argument list, but it was pretty
straightforward.
What does your implementation do with something like
f() + g():
...
? (A syntax error, I would hope.)
While no doubt it can be done, I still don't like the
idea very much. It seems like a violation of modularity
in the grammar, so to speak. The syntax effectively
allowed for the expression is severely limited by the
fact that a block follows it, which is a kind of backward
effect that violates the predominantly LL-flavour of
the rest of the syntax. There's a backward effect in
the semantics, too -- you can't properly understand what
the otherwise-normal-looking function call is doing
without knowing what comes later.
An analogy has been made with the insertion of self
into the arguments of a method. But that is something quite
different. In x.meth(y), the rules are being followed
quite consistently: the result of x.meth is being
called with y (and only y!) as an argument; the insertion
of self happens later.
But here, insertion of the thunk would occur *before* any
call was made at all, with no clue from looking at the
call itself.
Requiring arguments other than the block to be dealt with by currying
can lead to problems. I won't claim these problems are serious, but
they will be annoying.
You have some valid concerns there. You've given me
something to think about.
Here's another idea. Don't write the parameters in the form
of a call at all; instead, do this:
with_file foo.txt, w as f:
f.write(Spam!)
This would have the benefit of making it look more like
a control structure and less like a funny kind of call.
I can see some problems with that, though. Juxtaposing two
expressions doesn't really work, because the result can end up
looking like a function call or indexing operation. I
don't want to put a keyword in between because that would
mess up how it reads. Nor do I want to put some arbitrary
piece of punctuation in there.
The best I can think of right now is
with_file {foo.txt, w} as f:
f.write(Spam!)
If you google filetype:rb yield, you can see many the uses of yield in
ruby.
I'm sure that use cases can be found, but the pertinent question
is whether a substantial number of those use cases from Ruby fall
into the class of block-uses which aren't covered by other Python
facilities.
Also, I have a gut feeling that it's a bad idea to try to provide
for this. I think the reason is this: We're trying to create
something that feels like a user-defined control structure with
a suite, and there's currently no concept in Python of a suite
returning a value to be consumed by its containing control
structure. It would be something new, and it would require some
mental gymnastics to understand what it was doing. We already
have return and yield; this would be a third similar-yet-
different thing.
If it were considered important enough, it could easily be added
later, without disturbing anything. But I think it's best left
out of an initial specification.
--
Greg Ewing, Computer Science Dept, +--+
University of Canterbury, | A citizen of NewZealandCorp, a |
Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. |
[EMAIL PROTECTED] +--+
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
[Greg Ewing] My current thought is that it should look like this: with_file(filename) as f: do_something_with(f) The success of this hinges on how many use cases can be arranged so that the word 'as' makes sense in that position. [...] This way, the syntax is just expr ['as' assignment_target] ':' suite and the expr is evaluated quite normally. Perhaps it could be even simpler: [assignment_target '=']* expr ':' suite This would just be an extension of the regular assignment statement. (More in a longer post I'm composing off-line while picking cherries off the thread.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
In case my point about the difference between thunks and other callables (specifically decorators) slipped by, consider the documentation for staticmethod, which takes a callable. All the staticmethod documentation says about that callable's parameters is: A static method does not receive an implicit first argument Pretty simple I'd say. Or classmethod: A class method receives the class as implicit first argument, just like an instance method receives the instance. Again, pretty simple. Why are these simple? Because decorators generally pass on pretty much the same arguments as the callables they wrap. My point was just that because thunks don't wrap other normal callables, they can't make such abbreviations. You've got the special-casing backwards. It's not thinks that are special, but staticmethod (and decorators in general) because they take *any* callable. That's unusual -- most callable arguments have a definite signature, think of map(), filter(), sort() and Button callbacks. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
It strikes me that with something like this lexical declaration, we could abuse decorators as per Carl Banks's recipe[1] to get the equivalent of thunks: abuse being the operative word. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On 4/21/05, Guido van Rossum [EMAIL PROTECTED] wrote: It strikes me that with something like this lexical declaration, we could abuse decorators as per Carl Banks's recipe[1] to get the equivalent of thunks: abuse being the operative word. Yup. I was just drawing the parallel between: @withfile(readme.txt) def thunk(fileobj): for line in fileobj: print line and @withfile(readme.txt): # called by withfile as thunk(fileobj=file object) for line in fileobj: print line STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Greg Ewing wrote: I also have a thought concerning whether the block argument to the function should come first or last or whatever. My solution is that the function should take exactly *one* argument, which is the block. Any other arguments are dealt with by currying. In other words, with_file above would be defined as def with_file(filename): def func(block): f = open(filename) try: block(f) finally: f.close() return func This would also make implementation much easier. The parser isn't going to know that it's dealing with anything other than a normal expression statement until it gets to the 'as' or ':', by which time going back and radically re-interpreting a previous function call could be awkward. I made an example implementation, and this wasn't an issue. It took some code to stick the thunk into the argument list, but it was pretty straightforward. The syntax that is actually used by the parser can be the same regardless of whether or not argument list augmentation is done, so the parser will not find one more awkward than the other. This way, the syntax is just expr ['as' assignment_target] ':' suite and the expr is evaluated quite normally. Requiring arguments other than the block to be dealt with by currying can lead to problems. I won't claim these problems are serious, but they will be annoying. Say, for example, you create a block-accepting function that takes no arguments. Naturally, you would define it like this: def f(block): do_something_with_block Now, say you want to add to this function an optional argument, so you wrap another function around it like in your 'with_file' example above. Unfortunately, now you need to go find every call of this function and add empty parentheses. This is annoying. Remember the first time you added optional arguments to a function and what a relief it was not to have to go find every call to that function and stick in the extra argument? Those days are over! (well, in this case anyway.) Some people, aware of this problem of adding optional arguments, will define *all* of their block-accepting functions so that they are wrapped in another function, even if that function takes no arguments (and wars, annoying ones, will be fought over whether this is the right way to do it or not!): def f(): def real_func(block): pass return real_func Now the documentation gets confusing. Just saying that the function doesn't take any non-block arguments isn't enough. You would need very specific language, which many library authors will not provide. And there will always be that extra step in thought: do I need the stupid parentheses or not? There will inevitably be people (including me) who get the parentheses wrong because of absentmindedness or carelessness. This will be an extra little speed bump. Now, you may say that all these arguments apply to function decorators, so why have none of these problems appeared? The difference is that defining a function takes a long time, so a little speed bump when decorating it isn't a big deal. But blocks can be defined almost instantly. Much of their purpose has to do with making things quicker. Speed bumps are therefore a bigger deal. This will also be an issue for beginners who use python. A beginner won't necessarily have a good understanding of a function that returns a function. But such an understanding would be required simply to *use* block-accepting functions. Otherwise it would be completely mysterious why sometimes one sees this f(a,b,c) as i: pass and sometimes this g as i: pass even though both of these cases just seem to call the function that appears next to 'as' (imagine you don't have the source of 'f' and 'g'). Even worse, imagine finally learning the rule that parentheses are not allowed if there are zero arguments, and then seeing: h() as i: pass Now it would just seem arbitrary whether or not parentheses are required or disallowed. Such an issue may seem trivial to an experienced programmer, but can be very off-putting for a beginner. Another set of question arose for me when Barry started musing over the combination of blocks and decorators. What are blocks? Well, obviously they are callable. What do they return? The local namespace they created/modified? I think the return value of a block should be None. In constructs like with_file, the block is being used for its side effect, not to compute a value for consumption by the block function. I don't see a great need for blocks to be able to return values. If you google filetype:rb yield, you can see many the uses of yield in ruby. By looking for the uses in which yield's return value is used, you can find blocks that return values. For example, t = yield() or unless yield() indicate that a block is returning a value. It is true that most of the time blocks do not return values, but
Re: [Python-Dev] anonymous blocks
On Thu, 21 Apr 2005, Guido van Rossum wrote:
Perhaps it could be even simpler:
[assignment_target '=']* expr ':' suite
This would just be an extension of the regular assignment statement.
It sounds like you are very close to simply translating
expression... function_call(args):
suite
into
expression... function_call(args)(suitefunc)
If i understand what you proposed above, you're using assignment
as a special case to pass arguments to the inner suite, right? So:
inner_args = function_call(outer_args):
suite
becomes:
def suitefunc(inner_args):
suite
function_call(outer_args)(suitefunc)
?
This could get a little hard to understand if the right-hand side
of the assignment is more complex than a single function call.
I think the meaning would be unambiguous, just non-obvious. The
only interpretation i see for this:
x = spam('foo') + eggs('bar'):
suite
is this:
def suitefunc(x):
suite
spam('foo') + eggs('bar')(suitefunc)
but that could seem a little too mysterious. Or you could (in a
later compiler pass) forbid more complex expressions on the RHS.
On another note, would there be any difference between
x = spam():
suite
and
x = spam:
suite
?
-- ?!ng
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Guido van Rossum wrote:
I've been thinking about this a lot, but haven't made much
progess. Here's a brain dump.
I've been thinking about integrating PEP 325 (Resource-Release Support
for Generators) into the for-loop code, so that you could replace
[SNIP - using 'for' syntax to delineate the block and resource]
So I think that this is probably not the right thing to pursue,
I totally agree with your reasoning on this.
and we
might be better off with something along the lines of PEP 310. The
authors of PEP 310 agree; under Open Issues they wrote:
There are some simiralities in concept between 'with ...' blocks
and generators, which have led to proposals that for loops could
implement the with block functionality[3]. While neat on some
levels, we think that for loops should stick to being loops.
(Footnote [3] references the tread that originated PEP 325.)
Perhaps the most important lesson we've learned in this thread is that
the 'with' keyword proposed in PEP 310 is redundant -- the syntax
could just be
[VAR '=']* EXPR ':'
BODY
IOW the regular assignment / expression statement gets an optional
colon-plus-suite at the end.
Sure, but is the redundancy *that* bad? You should be able to pick up visually
that something is an anonymous block from the indentation but I don't know how
obvious it would be.
Probably, in the end, this minimal syntax would be fine, but it just seems
almost too plain in terms of screaming at me that something special is going on
there (the '=' in an odd place just quite cut if for me for my meaning of
special).
So now let's assume we accept PEP 310 with this change. Does this
leave any use cases for anonymous blocks uncovered? Ruby's each()
pattern is covered by generators; personally I prefer Python's
for var in seq: ...
over Ruby's much-touted
seq.each() {|var| ...}
The try/finally use case is covered by PEP 310. (If you want to
combine this with a for-loop in a single operation, you'll need PEP
325.)
The use cases where the block actually returns a value are probably
callbacks for things like sort() or map(); I have to admit that I'd
rather keep lambda for these (and use named functions for longer
blocks) than introduce an anonymous block syntax that can return
values! I also note that if you *already* have a comparison function,
Ruby's Array sort method doesn't let you pass it in as a function
argument; you have to give it a block that calls the comparison
function, because blocks are not the same as callables (and I'm not
sure that Ruby even *has* callables -- everything seems to be a
block).
My tentative conclusion remains: Python doesn't need Ruby blocks.
Brian Sabbey ought to come up with more examples rather than arguments
why his preferred syntax and semantics are best.
I think I agree with Samuele that it would be more pertinent to put all of this
effort into trying to come up with some way to handle cleanup in a generator.
-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks (don't combine them with generator finalization)
Guido van Rossum [EMAIL PROTECTED] wrote: [Brett] I think I agree with Samuele that it would be more pertinent to put all of this effort into trying to come up with some way to handle cleanup in a generator. I.e. PEP 325. But (as I explained, and you agree) that still doesn't render PEP 310 unnecessary, because abusing the for-loop for implied cleanup semantics is ugly and expensive, and would change generator semantics; and it bugs me that the finally clause's reachability depends on the destructor executing. Yes and no. PEP 325 offers a method to generators that handles cleanup if necessary and calls it close(). Obviously calling it close is a mistake. Actually, calling it anything is a mistake, and trying to combine try/finally handling in generators with __exit__/close (inside or outside of generators) is also a mistake. Start by saying, If a non-finalized generator is garbage collected, it will be finalized. Whether this be by an exception or forcing a return, so be it. If this were to happen, we have generator finalization handled by the garbage collector, and don't need to translate /any/ for loop. As long as the garbage collection requirement is documented, we are covered (yay!). What about ... i.__enter__() try: ... finally: i.__exit__() ... types of things? Well, you seem to have offered a syntax ... [VAR '=']* EXPR: BODY ... which seems to translate into ... [VAR = ] __var = EXPR try: BODY finally: __var.__exit__() ... or something like that. Great! We've got a syntax for resource allocation/freeing outside of generators, and a non-syntax for resource allocation/freeing inside of generators. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks (don't combine them with generator finalization)
On Apr 21, 2005, at 8:59 PM, Josiah Carlson wrote: Guido van Rossum [EMAIL PROTECTED] wrote: [Brett] I think I agree with Samuele that it would be more pertinent to put all of this effort into trying to come up with some way to handle cleanup in a generator. I.e. PEP 325. But (as I explained, and you agree) that still doesn't render PEP 310 unnecessary, because abusing the for-loop for implied cleanup semantics is ugly and expensive, and would change generator semantics; and it bugs me that the finally clause's reachability depends on the destructor executing. Yes and no. PEP 325 offers a method to generators that handles cleanup if necessary and calls it close(). Obviously calling it close is a mistake. Actually, calling it anything is a mistake, and trying to combine try/finally handling in generators with __exit__/close (inside or outside of generators) is also a mistake. Start by saying, If a non-finalized generator is garbage collected, it will be finalized. Whether this be by an exception or forcing a return, so be it. If this were to happen, we have generator finalization handled by the garbage collector, and don't need to translate /any/ for loop. As long as the garbage collection requirement is documented, we are covered (yay!). Well, for the CPython implementation, couldn't you get away with using garbage collection to do everything? Maybe I'm missing something.. import weakref class ResourceHandle(object): def __init__(self, acquire, release): acquire() # if I understand correctly, this is safer than __del__ self.ref = weakref.ref(self, lambda o:release()) class FakeLock(object): def acquire(self): print acquired def release(self): print released def with_lock(lock): r = ResourceHandle(lock.acquire, lock.release) yield None del r x = with_lock(FakeLock()) del x with_lock(FakeLock()).next() acquired released for ignore in with_lock(FakeLock()): ... print ignore ... acquired None released I could imagine someone complaining about generators that are never used missing out on the acquire/release. That could be solved with a trivial rewrite: def with_lock(lock): def _with_lock(r): yield None del r return _with_lock(ResourceHandle(lock.acquire, lock.release)) x = with_lock(FakeLock()) acquired del x released Of course, this just exaggerates Guido's it bugs me that the finally clause's reachability depends on the destructor executing.. but it does work, in CPython. It seems to me that this pattern would be painless enough to use without a syntax change... -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On Thu, Apr 21, 2005, Guido van Rossum wrote: Perhaps the most important lesson we've learned in this thread is that the 'with' keyword proposed in PEP 310 is redundant -- the syntax could just be [VAR '=']* EXPR ':' BODY IOW the regular assignment / expression statement gets an optional colon-plus-suite at the end. Yes, it could. The question then becomes whether it should. Because it's easy to indent Python code when you're not using a block (consider function calls with lots of args), my opinion is that like the optional colon after ``for`` and ``if``, the resource block *should* have a keyword. -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death. --GvR ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Guido or perhaps even (making for VAR optional in the for-loop syntax) Guido with Guido in synchronized(the_lock): Guido BODY This could be a new statement, so the problematic issue of implicit try/finally in every for statement wouldn't be necessary. That complication would only be needed for the above form. (Of course, if you've dispensed with this I am very likely missing something fundamental.) Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Ka-Ping Yee wrote: It seems to me that, in general, Python likes to use keywords for statements and operators for expressions. Probably worth noting that 'for', 'in' and 'if' in generator expressions and list comprehensions blur this distinction somewhat... Steve -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Guido van Rossum wrote: [Brett] I think I agree with Samuele that it would be more pertinent to put all of this effort into trying to come up with some way to handle cleanup in a generator. I.e. PEP 325. But (as I explained, and you agree) that still doesn't render PEP 310 unnecessary, because abusing the for-loop for implied cleanup semantics is ugly and expensive, and would change generator semantics; Right, I'm not saying PEP 310 shouldn't also be considered. It just seems like we are beginning to pile a lot on this discussion by bringing in PEP 310 and PEP 325 in at the same time since, as pointed out, there is no guarantee that anything will be called in a generator and thus making PEP 310 work in generators does not seem guaranteed to solve that problem (although I might have missed something; just started really following the thread today). At this point anonymous blocks just don't seem to be happening, at least not like in Ruby. Fine, I didn't want them anyway. Now we are trying to simplify resource cleanup and handling. What I am trying to say is that generators differ just enough as to possibly warrant a separate discussion from all of this other resource handling stuff. So I am advocating a more focused generator discussion since resource handling in generators is much more difficult than the general case in non-generator situations. I mean obviously in the general case all of this is handled already in Python today with try/finally. But with generators you have to jump through some extra hoops to get similar support (passing in anything that needs to be cleaned up, hoping that garbage collection will eventually handle things, etc.). and it bugs me that the finally clause's reachability depends on the destructor executing. Yeah, I don't like it either. I would rather see something like: def gen(): FILE = open(stuff.txt, 'rU') for line in FILE: yield line cleanup: FILE.close() and have whatever is in the 'cleanup' block be either accessible from a method in the generator or have it become the equivalent of a __del__ for the generator, or maybe even both (which would remove contention that whatever needs to be cleaned up is done too late thanks to gc not guaranteeing immediate cleanup). This way you get the guaranteed cleanup regardless and you don't have to worry about creating everything outside of the generator, passing it in, and then handling cleanup in a try/finally that contains the next() calls to the generator (or any other contortion you might have to go through). Anyway, my random Python suggestion for the day. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks (don't combine them with generator finalization)
Bob Ippolito wrote: On Apr 21, 2005, at 8:59 PM, Josiah Carlson wrote: Guido van Rossum [EMAIL PROTECTED] wrote: [Brett] I think I agree with Samuele that it would be more pertinent to put all of this effort into trying to come up with some way to handle cleanup in a generator. I.e. PEP 325. But (as I explained, and you agree) that still doesn't render PEP 310 unnecessary, because abusing the for-loop for implied cleanup semantics is ugly and expensive, and would change generator semantics; and it bugs me that the finally clause's reachability depends on the destructor executing. Yes and no. PEP 325 offers a method to generators that handles cleanup if necessary and calls it close(). Obviously calling it close is a mistake. Actually, calling it anything is a mistake, and trying to combine try/finally handling in generators with __exit__/close (inside or outside of generators) is also a mistake. Start by saying, If a non-finalized generator is garbage collected, it will be finalized. Whether this be by an exception or forcing a return, so be it. If this were to happen, we have generator finalization handled by the garbage collector, and don't need to translate /any/ for loop. As long as the garbage collection requirement is documented, we are covered (yay!). Well, for the CPython implementation, couldn't you get away with using garbage collection to do everything? Maybe I'm missing something.. [SNIP] Well, if you are missing something then so am I since your suggestion is basically correct. The only issue is that people will want more immediate execution of the cleanup code which gc cannot guarantee. That's why the ability to call a method with the PEP 325 approach gets rid of that worry. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks (don't combine them with generator finalization)
On Apr 22, 2005, at 12:28 AM, Brett C. wrote: Bob Ippolito wrote: On Apr 21, 2005, at 8:59 PM, Josiah Carlson wrote: Guido van Rossum [EMAIL PROTECTED] wrote: [Brett] I think I agree with Samuele that it would be more pertinent to put all of this effort into trying to come up with some way to handle cleanup in a generator. I.e. PEP 325. But (as I explained, and you agree) that still doesn't render PEP 310 unnecessary, because abusing the for-loop for implied cleanup semantics is ugly and expensive, and would change generator semantics; and it bugs me that the finally clause's reachability depends on the destructor executing. Yes and no. PEP 325 offers a method to generators that handles cleanup if necessary and calls it close(). Obviously calling it close is a mistake. Actually, calling it anything is a mistake, and trying to combine try/finally handling in generators with __exit__/close (inside or outside of generators) is also a mistake. Start by saying, If a non-finalized generator is garbage collected, it will be finalized. Whether this be by an exception or forcing a return, so be it. If this were to happen, we have generator finalization handled by the garbage collector, and don't need to translate /any/ for loop. As long as the garbage collection requirement is documented, we are covered (yay!). Well, for the CPython implementation, couldn't you get away with using garbage collection to do everything? Maybe I'm missing something.. [SNIP] Well, if you are missing something then so am I since your suggestion is basically correct. The only issue is that people will want more immediate execution of the cleanup code which gc cannot guarantee. That's why the ability to call a method with the PEP 325 approach gets rid of that worry. Well in CPython, if you are never assigning the generator to any local or global, then you should be guaranteed that it gets cleaned up at the right time unless it's alive in a traceback somewhere (maybe you WANT it to be!) or some insane trace hook keeps too many references to frames around.. It seems *reasonably* certain that for reasonable uses this solution WILL clean it up optimistically. -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On 4/19/05, Brian Sabbey [EMAIL PROTECTED] wrote: Guido van Rossum wrote: @acquire(myLock): code code code It would certainly solve the problem of which keyword to use! :-) And I think the syntax isn't even ambiguous -- the trailing colon distinguishes this from the function decorator syntax. I guess it would morph '@xxx' into user-defined-keyword. Hmm, this looks to me like a natural extension of decorators. Whether that is a good or a bad thing, I'm unable to decide :-) [I can think of a number of uses for it, PEP 310-style with-blocks being one, but I can't decide if lots of potential uses is too close to lots of potential for abuse :-)] How would acquire be defined? I guess it could be this, returning a function that takes a callable as an argument just like other decorators: def acquire(aLock): def acquirer(block): aLock.acquire() try: block() finally: aLock.release() return acquirer It really has to be this, IMO, otherwise the parallel with decorators becomes confusing, rather than helpful. and the substitution of @EXPR: CODE would become something like def __block(): CODE EXPR(__block) The question of whether assignments within CODE are executed within a new namespace, as this implies, or in the surrounding namespace, remains open. I can see both as reasonable (new namespace = easier to describe/understand, more in line with decorators, probably far easier to implement; surrounding namespace = probably more useful/practical...) Why not have the block automatically be inserted into acquire's argument list? It would probably get annoying to have to define inner functions like that every time one simply wants to use arguments. If this syntax is to be considered, in my view it *must* follow established decorator practice - and that includes the define-an-inner-function-and-return-it idiom. Of course, augmenting the argument list in that way would be different than the behavior of decorators as they are now. Exactly. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On Apr 20, 2005, at 5:43 AM, Paul Moore wrote: and the substitution of @EXPR: CODE would become something like def __block(): CODE EXPR(__block) The question of whether assignments within CODE are executed within a new namespace, as this implies, or in the surrounding namespace, remains open. I can see both as reasonable (new namespace = easier to describe/understand, more in line with decorators, probably far easier to implement; surrounding namespace = probably more useful/practical...) If it was possible to assign to a variable to a variable bound outside your function, but still in your lexical scope, I think it would fix this issue. That's always something I've thought should be possible, anyways. I propose to make it possible via a declaration similar to 'global'. E.g. (stupid example, but it demonstrates the syntax): def f(): count = 0 def addCount(): lexical count count += 1 assert count == 0 addCount() assert count == 1 Then, there's two choices for the block decorator: either automatically mark all variable names in the immediately surrounding scope lexical, or don't. Both of those choices are still consistent with the block just being a normal function, which I think is an important attribute. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Aahz wrote: On Tue, Apr 19, 2005, Shane Holloway (IEEE) wrote: However, my opinion is that it does not read smoothly. This form requires that I say what I'm doing with something before I know the context of what that something is. For me, blocks are not about shortening the code, but rather clarifying *intent*. H How is this different from defining functions before they're called? It's not. In a function scope I'd prefer to read top-down. When I write classes, I tend to put the public methods at the top. Utility methods used by those entry points are placed toward the bottom. In this way, I read the context of what I'm doing first, and then the details of the internal methods as I need to understand them. Granted I could achieve this effect with:: class Before: def readIt(self, filename): def readIt(): withFile(filename, doReading) def doReading(aFile): self.readPartA(aFile) self.readPartB(aFile) self.readPartC(aFile) return readIt() Which is fine with me, but the *intent* is more obfuscated than what the block construct offers. And I don't think my crew would appreciate if I did this very often. ;) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
[Shane Holloway] When I write classes, I tend to put the public methods at the top. Utility methods used by those entry points are placed toward the bottom. In this way, I read the context of what I'm doing first, and then the details of the internal methods as I need to understand them. Granted I could achieve this effect with:: class Before: def readIt(self, filename): def readIt(): withFile(filename, doReading) def doReading(aFile): self.readPartA(aFile) self.readPartB(aFile) self.readPartC(aFile) return readIt() Which is fine with me, but the *intent* is more obfuscated than what the block construct offers. And I don't think my crew would appreciate if I did this very often. ;) I typically solve that by making doReading() a method: class Before: def readit(self, filename): withFile(filename, self._doReading) def _doReading(self, aFile): self.readPartA(aFile) self.readPartB(aFile) self.readPartC(aFile) Perhaps not as Pure, but certainly Practical. :-) And you could even use __doReading to make it absolutely clear that doReading is a local artefact, if you care about such things. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Alex Martelli wrote: def withfile(filename, mode='r'): openfile = open(filename, mode) try: block(thefile=openfile) finally: openfile.close() i.e., let the block take keyword arguments to tweak its namespace I don't think I like that idea, because it means that from the point of view of the user of withfile, the name 'thefile' magically appears in the namespace without it being obvious where it comes from. (but assignments within the block should still affect its _surrounding_ namespace, it seems to me...). I agree with that much. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Josiah Carlson wrote: I once asked Any other use cases for one of the most powerful features of Ruby, in Python? I have yet to hear any sort of reasonable response. Why am I getting no response to my question? Either it is because I am being ignored, or no one has taken the time to translate one of these 'killer features' from Smalltalk or Ruby, or perhaps such translations show that there is a better way in Python already. My feeling is that it's the latter. I don't know about Ruby, but in Smalltalk, block-passing is used so heavily because it's the main way of implementing control structures there. While-loops, for-loops, even if-then-else, are not built into the language, but are implemented by methods that take block parameters. In Python, most of these are taken care of by built-in statements, or various uses of iterators and generators. There isn't all that much left that people want to do on a regular basis. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Steven Bethard wrote: Of course, even with the unpack list, you still have to know what kind of arguments the function calls your block with. And because these only appear within the code, e.g. block(openfile) you can't rely on easily accessible things like the function's signature. You can't rely on a function's signature alone to tell you much in any case. A distressingly large number of functions found in third-party extension modules have a help() string that just says something like fooble(arg,...) There's really no substitute for a good docstring! -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Shane Holloway (IEEE) wrote: class After: def readIt(self, filename): withFile(filename): self.readPartA(aFile) self.readPartB(aFile) self.readPartC(aFile) In my opinion, this is much smoother to read. This particular example brings up the question of how arguments like aFile get passed and named into the block. I anticipate the need for a place to put an argument declaration list. ;) My current thought is that it should look like this: with_file(filename) as f: do_something_with(f) The success of this hinges on how many use cases can be arranged so that the word 'as' makes sense in that position. What we need is a corpus of use cases so we can try out different phrasings on them and see what looks the best for the most cases. I also have a thought concerning whether the block argument to the function should come first or last or whatever. My solution is that the function should take exactly *one* argument, which is the block. Any other arguments are dealt with by currying. In other words, with_file above would be defined as def with_file(filename): def func(block): f = open(filename) try: block(f) finally: f.close() return func This would also make implementation much easier. The parser isn't going to know that it's dealing with anything other than a normal expression statement until it gets to the 'as' or ':', by which time going back and radically re-interpreting a previous function call could be awkward. This way, the syntax is just expr ['as' assignment_target] ':' suite and the expr is evaluated quite normally. Another set of question arose for me when Barry started musing over the combination of blocks and decorators. What are blocks? Well, obviously they are callable. What do they return? The local namespace they created/modified? I think the return value of a block should be None. In constructs like with_file, the block is being used for its side effect, not to compute a value for consumption by the block function. I don't see a great need for blocks to be able to return values. How do blocks work with control flow statements like break, continue, yield, and return? Perhaps break and continue raise exceptions similar to StopIteration in this case? Something like that, yes. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Greg Ewing wrote: Steven Bethard wrote: Of course, even with the unpack list, you still have to know what kind of arguments the function calls your block with. And because these only appear within the code, e.g. block(openfile) you can't rely on easily accessible things like the function's signature. You can't rely on a function's signature alone to tell you much in any case. A distressingly large number of functions found in third-party extension modules have a help() string that just says something like fooble(arg,...) There's really no substitute for a good docstring! True enough. But the point still stands. Currently, if we describe a function's input (parameters) and output (return value), we can basically fully document the function (given a thorough enough description of course).[1] Functions that accept thunks/blocks require documentation for an additional piece of information that is not part of the input or output of the function: the parameters with which the thunk/block is called. So while: fooble(arg) is pretty nasty, documentation that tells me that 'arg' is a string is probably enough to set me on the right track. But if the documentation tells me that arg is a thunk/block, that's almost certainly not enough to get me going. I also need to know how that thunk/block will be called. True, if arg is not a thunk/block, but another type of callable, I may still need to know how it will be called. But I think with non thunks/blocks, there are a lot of cases where this is not necessary. Consider the variety of decorator recipes.[2] Most don't document what parameters the wrapped function will be called with because they simply pass all arguments on through with *args and **kwargs. Thus the wrapped function will take the same parameters as the original function did. Or if they're different, they're often a relatively simple modification of the original function's parameters, ala classmethod or staticmethod. But thunks/blocks don't work this way. They're not wrapping a function that already takes arguments. They're wrapping a code block that doesn't. So they certainly can't omit the parameter description entirely, and they can't even describe it in terms of a modification to an already existing set of parameters. Because the parameters passed from a thunk/block-accepting function to a thunk are generated by the function itself, all the parameter documentation must be contained within the thunk/block-accepting function. It's not like it's the end of the world of course. ;-) I can certainly learn to document my thunks/blocks thoroughly. I just think it's worth noting that there *would* be a learning process because there are additional pieces of information I'm not used to having to document. STeVe [1] I'm ignoring the issue of functions that modify parameters or globals, but this would also be required for thunks/blocks, so I don't think it detracts from the argument. [2] Probably worth noting that a very large portion of the functions I've written that accepted other functions as parameters were decorators. I lean towards a fairly OO style of programming, so I don't pass around a lot of callbacks. Presumably someone who relies heavily on callbacks would be much more used to documenting the parameters with which a function is called. Still, I think there is probably a large enough group that has similar style to mine that my argument is still valid. -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
(I apologize that this is my first post. Please don't flame me into
oblivion or think I'm a quack!)
(Having met JJ I can assure he's not a quack. But don't let that stop
the flames. :-)
Have you guys considered the following syntax for anonymous blocks? I
think it's possible to parse given Python's existing syntax:
items.doFoo(
def (a, b) {
return a + b
},
def (c, d) {
return c + d
}
)
Notice the trick is that there is no name between the def and the (,
and the ) is followed by a {.
I understand that there is hesitance to use {}. However, you can
think of this as a Python special case on the same level as using ;
between statements on a single line. From that perspective, it's not
inconsistent at all.
It would be a lot less inconsistent if {...} would be acceptable
alternative block syntax everywhere.
But what exactly are you trying to accomplish here? I think that
putting the defs *before* the call (and giving the anonymous blocks
temporary local names) actually makes the code clearer:
def block1(a, b):
return a + b
def block2(c, d):
return c + d
items.doFoo(block1, block2)
This reflects a style pattern that I've come to appreciate more
recently: when breaking a call with a long argument list to fit on
your screen, instead of trying to find the optimal break points in the
argument list, take one or two of the longest arguments and put them
in local variables. Thus, instead of this:
self.disentangle(0x40,
self.triangulation(The quick brown fox jumps over
the lazy dog),
self.indent+1)
I'd recommend this:
tri = self.subcalculation(The quick brown fox jumps over the lazy dog)
self.disentangle(0x40, tri, self.indent+1)
IMO this is clearer, and even shorter!
If we apply this to the anonymous block problem, we may end up finding
lambda the ultimate compromise -- like a gentleman in the back of my
talk last week at baypiggies observed (unfortunately I don't know his
name).
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Shannon -jj Behrens wrote:
Have you guys considered the following syntax for anonymous blocks? I
think it's possible to parse given Python's existing syntax:
items.doFoo(
def (a, b) {
return a + b
},
def (c, d) {
return c + d
}
)
There was a proposal in the last few days on comp.lang.python that allows
you to do this in a way that requires less drastic changes to python's
syntax. See the thread pre-PEP: Suite-Based Keywords (shamless plug)
(an earlier, similar proposal is here:
http://groups.google.co.uk/groups?selm=mailman.403.1105274631.22381.python-list
%40python.org ).
In short, if doFoo is defined like:
def doFoo(func1, func2):
pass
You would be able to call it like:
doFoo(**):
def func1(a, b):
return a + b
def func2(c, d):
return c + d
That is, a suite can be used to define keyword arguments.
-Brian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
See the thread pre-PEP: Suite-Based Keywords (shamless plug) (an earlier, similar proposal is here: http://groups.google.co.uk/groups?selm=mailman.403.1105274631.22381.python-list %40python.org ). In short, if doFoo is defined like: def doFoo(func1, func2): pass You would be able to call it like: doFoo(**): def func1(a, b): return a + b def func2(c, d): return c + d That is, a suite can be used to define keyword arguments. I'm still not sure how this is particularly solving a pressing problem that isn't solved by putting the function definitions in front of the call. I saw the first version of the proto-PEP and didn't think that the motivating example (keeping the getx/setx methods passed to a property definition out of the class namespace) was all that valuable. Two more issues: (1) It seems that *every* name introduced in the block automatically becomes a keyword argument. This looks like a problem, since you could easily need temporary variables there. (I don't see that a problem with class bodies because the typical use there is only method and property definitions and the occasional instance variable default.) (2) This seems to be attaching a block to a specific function call but there are more general cases: e.g. you might want to assign the return value of doFoo() to a variable, or you might want to pass it as an argument to another call. *If* we're going to create syntax for anonymous blocks, I think the primary use case ought to be cleanup operations to replace try/finally blocks for locking and similar things. I'd love to have syntactical support so I can write blahblah(myLock): code code code instead of myLock.acquire() try: code code code finally: myLock.release() -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
At 11:55 AM 04/19/2005 -0700, Guido van Rossum wrote: I'd recommend this: tri = self.subcalculation(The quick brown fox jumps over the lazy dog) self.disentangle(0x40, tri, self.indent+1) IMO this is clearer, and even shorter! What was your opinion on where as a lambda replacement? i.e. foo = bar(callback1, callback2) where: def callback1(x): print hello, def callback2(x): print world! I suspect that you like the define-first approach because of your tendency to ask questions first and read later. That is, you want to know what callback1 and callback2 are before you see them passed to something. However, other people seem to like to have the context first, then fill in the details of each callback later. Interestingly, this syntax also works to do decoration, though it's not a syntax that was ever proposed for that. e.g.: foo = classmethod(foo) where: def foo(cls,x,y,z): # etc. foo = property(get_foo,set_foo) where: def get_foo(self): # ... def set_foo(self): # ... I don't mind @decorators, of course, but maybe they wouldn't be needed here. If we apply this to the anonymous block problem, we may end up finding lambda the ultimate compromise -- like a gentleman in the back of my talk last week at baypiggies observed (unfortunately I don't know his name). -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/pje%40telecommunity.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
At 03:39 PM 04/19/2005 -0400, Phillip J. Eby wrote: I suspect that you like the define-first approach because of your tendency to ask questions first and read later. Oops; I forgot to put the smiley on that. It was supposed to be a humorous reference to a comment Guido made in private e-mail about the Dr. Dobbs article I wrote on decorators. He had said something similar about the way he reads articles, expecting the author to answer all his questions up front. Without that context, the above sentence sounds like some sort of snippy remark that I did not intend it to be. Sorry. :( ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On 4/19/05, Guido van Rossum [EMAIL PROTECTED] wrote: I'm still not sure how this is particularly solving a pressing problem that isn't solved by putting the function definitions in front of the Well. As to what I've read in my short python experience, people wants to change the language *not* because they have a problem that can not be solved in a different way, but because they *like* to solve it in a different way. And you, making a stand against this, are a main Python feature. .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Guido van Rossum wrote: See the thread pre-PEP: Suite-Based Keywords (shamless plug) (an earlier, similar proposal is here: http://groups.google.co.uk/groups?selm=mailman.403.1105274631.22381.python-list %40python.org ). In short, if doFoo is defined like: def doFoo(func1, func2): pass You would be able to call it like: doFoo(**): def func1(a, b): return a + b def func2(c, d): return c + d That is, a suite can be used to define keyword arguments. I'm still not sure how this is particularly solving a pressing problem that isn't solved by putting the function definitions in front of the call. I saw the first version of the proto-PEP and didn't think that the motivating example (keeping the getx/setx methods passed to a property definition out of the class namespace) was all that valuable. OK. I think most people (myself included) who would prefer to define properties (and event handlers, etc.) in this way are motivated by the perception that the current method is just ugly. I don't know that it solves any pressing problems. Two more issues: (1) It seems that *every* name introduced in the block automatically becomes a keyword argument. This looks like a problem, since you could easily need temporary variables there. (I don't see that a problem with class bodies because the typical use there is only method and property definitions and the occasional instance variable default.) Combining the suite-based keywords proposal with the earlier, 'where' proposal (linked in my above post), you would be able to name variables individually in the case that temporary variables are needed: f(x=x): x = [i**2 for i in [1,2,3]] (2) This seems to be attaching a block to a specific function call but there are more general cases: e.g. you might want to assign the return value of doFoo() to a variable, or you might want to pass it as an argument to another call. The 'where' proposal also doesn't have this problem. Any expression is allowed. *If* we're going to create syntax for anonymous blocks, I think the primary use case ought to be cleanup operations to replace try/finally blocks for locking and similar things. I'd love to have syntactical support so I can write blahblah(myLock): code code code instead of myLock.acquire() try: code code code finally: myLock.release() Well, that was my other proposal, pre-PEP: Simple Thunks (there is also an implementation). It didn't seem to go over all that well. I am going to try to rewrite it and give more motivation and explanation (and maybe use 'with' and 'from' instead of 'do' and 'in' as keywords). -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On Tue, 2005-04-19 at 15:24, Guido van Rossum wrote: *If* we're going to create syntax for anonymous blocks, I think the primary use case ought to be cleanup operations to replace try/finally blocks for locking and similar things. I'd love to have syntactical support so I can write blahblah(myLock): code code code instead of myLock.acquire() try: code code code finally: myLock.release() Indeed, it would be very cool to have these kind of (dare I say) block decorators for managing resources. The really nice thing about that is when I have to protect multiple resources in a safe, but clean way inside a single block. Too many nested try/finally's cause you to either get sloppy, or really ugly (or both!). RSMotD (random stupid musing of the day): so I wonder if the decorator syntax couldn't be extended for this kind of thing. @acquire(myLock): code code code -Barry signature.asc Description: This is a digitally signed message part ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Guido van Rossum wrote: tri = self.subcalculation(The quick brown fox jumps over the lazy dog) self.disentangle(0x40, tri, self.indent+1) IMO this is clearer, and even shorter! But it clutters the namespace with objects you don't need. So the complete equivalent would be more close to: tri = self.subcalculation(The quick brown fox jumps over the lazy dog) self.disentangle(0x40, tri, self.indent+1) del tri which seems a bit odd to me. If we apply this to the anonymous block problem, we may end up finding lambda the ultimate compromise -- like a gentleman in the back of my talk last week at baypiggies observed (unfortunately I don't know his name). It wasn't me ;-) It seems this keeps getting back at you. Wish I had thought of this argument before. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
@acquire(myLock): code code code It would certainly solve the problem of which keyword to use! :-) And I think the syntax isn't even ambiguous -- the trailing colon distinguishes this from the function decorator syntax. I guess it would morph '@xxx' into user-defined-keyword. How would acquire be defined? I guess it could be this, returning a function that takes a callable as an argument just like other decorators: def acquire(aLock): def acquirer(block): aLock.acquire() try: block() finally: aLock.release() return acquirer and the substitution of @EXPR: CODE would become something like def __block(): CODE EXPR(__block) I'm not yet sure whether to love or hate it. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Guido van Rossum wrote: @acquire(myLock): code code code It would certainly solve the problem of which keyword to use! :-) And I think the syntax isn't even ambiguous -- the trailing colon distinguishes this from the function decorator syntax. I guess it would morph '@xxx' into user-defined-keyword. How would acquire be defined? I guess it could be this, returning a function that takes a callable as an argument just like other decorators: def acquire(aLock): def acquirer(block): aLock.acquire() try: block() finally: aLock.release() return acquirer and the substitution of @EXPR: CODE would become something like def __block(): CODE EXPR(__block) Why not have the block automatically be inserted into acquire's argument list? It would probably get annoying to have to define inner functions like that every time one simply wants to use arguments. For example: def acquire(block, aLock): aLock.acquire() try: block() finally: aLock.release() @acquire(myLock): code code code Of course, augmenting the argument list in that way would be different than the behavior of decorators as they are now. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Why not have the block automatically be inserted into acquire's argument list? It would probably get annoying to have to define inner functions like that every time one simply wants to use arguments. But the number of *uses* would be much larger than the number of block decorators you'd be coding. If you find yourself writing new block decorators all the time that's probably a sign you're too much in love with the feature. :-) For example: def acquire(block, aLock): aLock.acquire() try: block() finally: aLock.release() @acquire(myLock): code code code Of course, augmenting the argument list in that way would be different than the behavior of decorators as they are now. I don't like implicit modifications of argument lists other than by method calls. It's okay for method calls because in the x.foo(a) == foo(x, a) equivalence, x is really close to the beginning of the argument list. And your proposal would preclude parameterless block decorators (or turn them into an ugly special case), which I think might be quite useful: @forever: infinite loop body @ignore: not executed at all @require: assertions go here and so on. (In essence, we're inventing the opposite of barewords in Perl here, right?) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Guido van Rossum wrote: As I said before, I'm not sure why keeping get_foo etc. out of the class namespace is such a big deal. In fact, I like having them there (sometimes they can even be handy, e.g. you might be able to pass the unbound get_foo method as a sort key). Not to mention that it's possible to override get_foo in subclasses if done right ... Two approaches are here: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/408713 Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
[Guido van Rossum] @EXPR: CODE would become something like def __block(): CODE EXPR(__block) I'm not yet sure whether to love or hate it. :-) Is it preferable for CODE to execute in its own namespace (the above being a literal translation of the given code), or for it to execute in the originally defined namespace? Deferring to Greg Ewing for a moment [1]: They should be lexically scoped, not dynamically scoped. Wrapped blocks in an old namespace, I believe, is the way to go, especially for things like... @synchronize(fooLock): a = foo.method() I cannot come up with any code for which CODE executing in its own namespace makes sense. Can anyone else? discussion on the overlap with PEP 310 removed for brevity - Josiah [1] http://mail.python.org/pipermail/python-dev/2005-March/052239.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
RSMotD (random stupid musing of the day): so I wonder if the decorator syntax couldn't be extended for this kind of thing. @acquire(myLock): code code code Would it be useful for anything other than mutex-locking? And wouldn't it be better to make a function of the block wrapped in a block-decorator and then use a normal decorator? -- mvh Björn ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On Tue, Apr 19, 2005 at 01:33:15PM -0700, Guido van Rossum wrote: @acquire(myLock): code code code It would certainly solve the problem of which keyword to use! :-) And I think the syntax isn't even ambiguous -- the trailing colon distinguishes this from the function decorator syntax. I guess it would morph '@xxx' into user-defined-keyword. How would acquire be defined? I guess it could be this, returning a function that takes a callable as an argument just like other decorators: [snip] and the substitution of @EXPR: CODE would become something like def __block(): CODE EXPR(__block) I'm not yet sure whether to love or hate it. :-) I don't know what the purpose of these things is, but I do think they should be like something else to avoid learning something new. Okay, I lied, I do know what these are: namespace decorators Namespaces are currently modules or classes, and decorators currently apply only to functions. The dissonance is that function bodies are evaluated later and namespaces (modules and classes) are evaluated immediately. I don't know if adding a namespace that is only evaluated later makes sense. It is only an extra case but it is one extra case to remember. At best I have only channeled Guido once, and by accident[1] so I'll stay out of the specifics (for a bit). -jackdied [1] during the decorator syntax bru-ha-ha at a Boston PIG meeting I suggested Guido liked the decorator-before-function because it made more sense in Dutch. I was kidding, but someone who knows a little Dutch (Deibel?) stated this was, in fact, the case. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On 4/19/05, BJörn Lindqvist [EMAIL PROTECTED] wrote: RSMotD (random stupid musing of the day): so I wonder if the decorator syntax couldn't be extended for this kind of thing. @acquire(myLock): code code code Would it be useful for anything other than mutex-locking? And wouldn't it be better to make a function of the block wrapped in a block-decorator and then use a normal decorator? Yes. Check how blocks in Smalltalk and Ruby are used for starters. Regards, Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
*If* we're going to create syntax for anonymous blocks, I think the primary use case ought to be cleanup operations to replace try/finally blocks for locking and similar things. I'd love to have syntactical support so I can write I heartily agree! Especially when you have very similar try/finally code you use in many places, and wish to refactor it into a common area. If this is done, you are forced into a callback form like follows:: def withFile(filename, callback): aFile = open(filename, 'r') try: result = callback(aFile) finally: aFile.close() return result class Before: def readIt(self, filename): def doReading(aFile): self.readPartA(aFile) self.readPartB(aFile) self.readPartC(aFile) withFile(filename, doReading) Which is certainly functional. I actually use the idiom frequently. However, my opinion is that it does not read smoothly. This form requires that I say what I'm doing with something before I know the context of what that something is. For me, blocks are not about shortening the code, but rather clarifying *intent*. With this proposed change, the code becomes:: class After: def readIt(self, filename): withFile(filename): self.readPartA(aFile) self.readPartB(aFile) self.readPartC(aFile) In my opinion, this is much smoother to read. This particular example brings up the question of how arguments like aFile get passed and named into the block. I anticipate the need for a place to put an argument declaration list. ;) And no, I'm not particularly fond of Smalltalk's solution with | aFile |, but that's just another opinion of aesthetics. Another set of question arose for me when Barry started musing over the combination of blocks and decorators. What are blocks? Well, obviously they are callable. What do they return? The local namespace they created/modified? How do blocks work with control flow statements like break, continue, yield, and return? I think these questions have good answers, we just need to figure out what they are. Perhaps break and continue raise exceptions similar to StopIteration in this case? As to the control flow questions, I believe those answers depend on how the block is used. Perhaps a few different invocation styles are applicable. For instance, the method block.suite() could return a tuple such as (returnedValue, locals()), where block.__call__() would simply return like any other callable. It would be good to figure out what the control flow difference is between:: def readAndReturn(self, filename): withFile(filename): a = self.readPartA(aFile) b = self.readPartB(aFile) c = self.readPartC(aFile) return (a, b, c) and:: def readAndReturn(self, filename): withFile(filename): a = self.readPartA(aFile) b = self.readPartB(aFile) c = self.readPartC(aFile) return (a, b, c) Try it with yield to further vex the puzzle. ;) Thanks for your time! -Shane Holloway ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
On 4/19/05, Alex Martelli [EMAIL PROTECTED] wrote:
Well, one obvious use might be, say:
@withfile('foo.bar', 'r'):
content = thefile.read()
but that would require the decorator and block to be able to interact
in some way, so that inside the block 'thefile' is defined suitably.
it be better to make a function of the block wrapped in a
block-decorator and then use a normal decorator?
From a viewpoint of namespaces, I think it would be better to have the
block execute in the same namespace as the code surrounding it, not a
separate one (assigning to 'content' would not work otherwise), so a
nested function would not be all that useful. The problem might be,
how does the _decorator_ affect that namespace. Perhaps:
def withfile(filename, mode='r'):
openfile = open(filename, mode)
try:
block(thefile=openfile)
finally:
openfile.close()
i.e., let the block take keyword arguments to tweak its namespace (but
assignments within the block should still affect its _surrounding_
namespace, it seems to me...).
I'm not a big fan of this means of tweaking the block's namespace. It
means that if you use a block decorator, you might find that names
have been 'magically' added to your namespace. This has a bad code
smell of too much implicitness to me...
I believe this was one of the reasons Brian Sabbey's proposal looked
something like:
do unpack_list in returnval = callable(params):
code
This way you could write the block above as something like:
def withfile(filename, mode='r'):
def _(block):
openfile = open(filename, mode)
try:
block(openfile)
finally:
openfile.close()
return _
do thefile in withfile('foo.bar', 'r'):
content = thefile.read()
where 'thefile' is explicitly named in the do/in-statement's unpack
list. Personally, I found the 'do' and 'in' keywords very confusing,
but I do like the fact that the parameters passed to the thunk/block
are expanded in an explicit unpack list. Using @, I don't see an easy
way to insert such an unpack list...
Of course, even with the unpack list, you still have to know what kind
of arguments the function calls your block with. And because these
only appear within the code, e.g.
block(openfile)
you can't rely on easily accessible things like the function's
signature. It means that unlike other callables that can basically
document parameters and return type, block decorators would have to
document parameters, return type, and the parameters with which they
call the block...
STeVe
--
You can wordify anything if you just verb it.
--- Bucky Katt, Get Fuzzy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
