Re: [Python-Dev] PEP 343 - Abstract Block Redux
Greg Ewing [EMAIL PROTECTED] writes: Guido van Rossum wrote: PEP 340 is still my favorite, but it seems there's too much opposition to it, I'm not opposed to PEP 340 in principle, but the ramifications seemed to be getting extraordinarily complicated, and it seems to be hamstrung by various backwards-compatibility constraints. E.g. it seems we can't make for-loops work the way they should in the face of generator finalisation or we break old code. I think I zoned this part of the discussion out, but I've written code like this: lineiter = iter(aFile) for line in lineiter: if sectionmarker in line: break parseSection1Line(line) for line in lineiter: if sectionmarker in line: break parseSection2Line(line) (though, not *quite* that regular...) This is, to me, neat and clear. I don't find the idea that iterators are tied to exactly 1 for loop an improvement (even though they usually will be). Cheers, mwh -- thirmite what's a web widget?? glyph thirmite: internet on a stick, on fire Acapnotic with web sauce! -- from Twisted.Quotes ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Phillip J. Eby wrote: At 08:21 PM 5/16/2005 -0400, Jack Diederich wrote: I still haven't gotten used to Guido's heart-attack inducing early enthusiasm for strange things followed later by a simple proclamation I like. Some day I'll learn that the sound of fingernails on the chalkboard is frequently followed by candy for the whole class. Heh. +1 for QOTW. :) Indeed. Particularly since it sounds like son-of-PEP-343 will be the union of PEP 310 and PEP 340 that I've been trying to figure out :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Paul Moore wrote: On 5/17/05, Nick Coghlan [EMAIL PROTECTED] wrote: Previously Belaboured Point? (Just guessing from context here, but if I'm right, that's one acronym I'm going to have to remember. . .) Practicality Beats Purity, surely...? D'oh! *slaps forehead* Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Nick Coghlan wrote: Wouldn't that mean we run the risk of suppressing a *real* SystemExit if it occurs while a generator is being finalised? Perhaps a new exception IteratorExit, which is a subclass of SystemExit. Then well-behaved code wouldn't trap it accidentally, and the finalisation code wouldn't? ... inadvertently suppress a real SystemExit. Cheers, Nick. Must have been distracted halfway through that sentence :) -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
[Guido van Rossum] I'm in favor of the general idea, but would like to separate the error injection and finalization API for generators into a separate PEP, which would then compete with PEP 288 and PEP 325. [Nick Coghlan] Without that it pretty much devolves into the current version of PEP 343, though (as far as I can tell, the two PEP's now agree on the semantics of with statements) But that's okay, right? We can just accept both PEPs and we're done. (And PEP 342 at the same time. :-) Discussing one PEP at a time is sometimes easier, even if they are meant to be used together. I think the API *should* be made public if it is available internally; I don't see any implementation reasons why it would simplify the implementation if it was only available internally. If it's internal, we don't need to name it immediately - working out the public API can then be decoupled from the ability to use generators to write resource managers. That's a minor point -- it's not like generators have a namespace for the user that we don't want to pollute, so we can pick any name we like. Here are some issues to resolve in such a PEP. - What does _inject_exception() return? It seems that it should raise the exception that was passed into it, but I can't find this written out explicitly. That's a good point. The intent was for it to be equivalent to the exception being reraised at the point of the last yield. At that point, control flows like it would for a call to next() with code inside the generator that looked like: yield exc_type, value, tb = _passed_in_exception() raise exc_type, value, tb So, also, _inject_exception() will appear to raise the exception that was passed to it, unless the generator catches it. - What should happen if a generator, in a finally or except clause reached from _inject_exception(), executes another yield? I'm tempted to make this a legitimate outcome (from the generator's perspective) and reserve it for some future statement that implements the looping semantics of PEP 340; the *_template decorator's wrapper class however should consider it an error, just like other protocol mismatches like not yielding the first time or yielding more than once in response to next(). Nick's code in fact does all this right, Ijust think it should be made explicit. Yep, that was the intent - you're correct that describing it in the text as well would make it clear that this is deliberate. OK. Please update the PEP then! - TerminateIteration is a lousy name, since terminate means about the same as stop, so there could be legitimate confusion with StopIteration. In PEP 340 I used StopIteration for this purpose, but someone explained this was a poor choice since existing generators may contain code that traps StopIteration for other purposes. Perhaps we could use SystemExit for this purpose? Pretty much everybody is supposed to let this one pass through since its purpose is only to allow cleanup upon program (or thread) exit. Wouldn't that mean we run the risk of suppressing a *real* SystemExit if it occurs while a generator is being finalised? D'oh. Yes. Perhaps a new exception IteratorExit, which is a subclass of SystemExit. Then well-behaved code wouldn't trap [SystemExit] accidentally, and the finalisation code wouldn't? Nah, I think it needs to be a brand spanking new exception. It just can't be called TerminateIteration. How about GeneratorFinalization to be utterly clear? - I really don't like reusing __del__ as the method name for any kind of destructor; __del__ has all sorts of special semantics (the GC treats objects with a __del__ method specially). I guess if file objects can live without a __del__ method to automatically close, generators that require finalisation can survive without it. Right. At the C level there is of course finalization (the tp_dealloc slot in the PyTypeObject struct) but it doesn't have a Python entry point to call it. And that's intentional, since the typical code executed by that slot *really* destroys the object and must only be called when the VM is absolutely sure that the object can't be reached in any other way. if it were callable from Python, that guarantee would be void. - The all_lines() example jars me. Somehow it bugs me that its caller has to remember to care about finalizing it. The alternative is to have some form of automatic finalisation in for loops, or else follow up on PJE's idea of making with statements allow pretty much *anything* to be used as VAR1. Automatic finalisation (like that now described in the Rejected Options section of my PEP) makes iterators/generators that manage resources 'just work' (nice) but complicates the semantics of generators (bad) and for loops (bad). The current version of my PEP means you need to know that a particular generator/iterator needs finalisation, and rearrange code to
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Nick Coghlan wrote: Paul Moore wrote: On 5/17/05, Nick Coghlan [EMAIL PROTECTED] wrote: Previously Belaboured Point? (Just guessing from context here, but if I'm right, that's one acronym I'm going to have to remember. . .) Practicality Beats Purity, surely...? D'oh! *slaps forehead* Cheers, Nick. Hmmm... looks like Google needs a Search Only in Python Terminology radio button... Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Brett C. wrote: Nick's was obviously directly against looping, but, with no offense to Nick, how many other people were against it looping? It never felt like it was a screaming mass with pitchforks but more of a I don't love it, but I can deal crowd. My problem with looping was that, with it, the semantics of a block statement would be almost, but not quite, exactly like those of a for-loop, which seems to be flying in the face of TOOWTDI. And if it weren't for the can't-finalise-generators-in-a-for-loop backward compatibility problem, the difference would be even smaller. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Ron Adam wrote: He should also be able to put try excepts before the yield, and after the yield, or in the block. (But not surrounding the yield, I think.) I was given to understand that yield is currently allowed in try-except, just not try-finally. So this would require a non-backwards-compatible change. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Steven Bethard wrote: If I've misunderstood, and there are other situations when needs_finish is required, it'd be nice to see some more examples. The problem is try/except/else blocks - those are currently legal, so the programmer has to make the call about whether finalisation is needed or not. I'll put this in the Open Issues section of the PEP - doing it lexically seems a little too magical for my taste (since it suddenly becomes more difficult to do partial iteration on the generator), but the decorator is a definite wart. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
On 5/15/05, Steven Bethard [EMAIL PROTECTED] wrote: Having done the python-dev summary on this topic, You have my deepest sympathy :-) So in some sense, PEP 340 was the reason for the lack of enthusiasm; with the semantics laid out, people were forced to deal with a specific implementation instead of a variety of wild suggestions. I'm not sure I agree with that - to me, PEP 340 felt like the consolidation of the previous discussion. My feeling was cool - we've had the discussion, now we've formalised the results, maybe a few details to tidy up and then we can see the implementation being checked in. Then Nick's proposal *failed* to feel like the tidying up of the details, and PEP 343 felt like giving up on the powerful (but hard) bits. It's all people's impressions, though, so maybe I'm just bitter and cynical :-) Interestingly, some new ideas have started appearing again (the GUI example someone raised yesterday, for instance). But with the current multiple PEPs situation, I can't evaluate such ideas, as I've no clue which of the various proposals would support them. No-one is looking at PEP 343, or Nick's PEP 3XX, and saying hey, that's neat - I can do XXX with that!. This makes me feel that we've thrown out the baby with the bathwater. I'd be surprised if you can find many examples that PEP 340 can do that PEP 3XX can't. In which cask, Nick is marketing it really badly - I hadn't got that impression at all. And if Nick's proposal really *is* PEP 340 with the issues people had resolved, how come Guido isn't supporting it? (By the way, I agree with Philip Eby - Nick's proposal really needs to be issued as a proper PEP - although if it's that close to just being a fix for PEP 340, it should probably just be the new version of PEP 340). (Yes, I know PEP 342 is integral to many of the neat features, but I get the impression that PEP 342 is being lost - later iterations of the other two PEPs are going out of their way to avoid assuming PEP 324 is implemented...) Not very far out of their way. Well, PEP 343 uses def template(): before yield after rather than def template(): before try: yield finally: after which I would argue is better - but it needs PEP 342 functionality. OTOH, Guido argues there are other reasons for the PEP 343 style. Also, the discussion has moved to resource objects with special methods rather than generators as templates - which I see as a direct consequence of PEP 342 being excluded. One of the things I really liked about PEP 340 was the generator template style of code, with yield as the block goes here placeolder If you want to get people enthused about PEP 342 again (which is the right way to make sure it gets excepted), what would really help is a bunch of good examples of how it could be used. In my view, *every* PEP 340/343/3XX example when written in generator form counts as a good example (see above). Neat coroutine tricks and the like are the additional benefits - maybe bordering on abuses in some cases, so I don't want to focus on them in case we get into arguments about whether a feature is bad simply because it *can* be abused... But the key use case, for me, *is* the generator-as-template feature. I'm convinced that such a tone is inevitable after 30 days and over 700 messages on *any* topic. ;-) Which is why I regret that Guido didn't just go ahead and implement it, consensus be damned :-) I vote for a dictatorship :-) Ok, back to summarizing this fortnight's 380+ PEP 340 messages. ;-) Best of luck - and in case you need a motivation boost, can I just say that this sort of thread is why I have the greatest respect and appreciation for the job the summarisers do. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
[Guido] In rev 1.10 I moved the __enter__ call out of the try-block again. Having it inside was insane: when __enter__ fails, it should do its own cleanup rather than expecting __exit__ to clean up after a partial __enter__. [Ka-Ping Yee] No, it wasn't insane. You had a good reason for putting it there. I did some introspection, and it was definitely a temporary moment of insanity: (1) I somehow forgot the obvious solution (that __enter__ should clean up its own mess if it doesn't make it to the finish line); (2) once a generator raises an exception it cannot be resumed, so the generator-based example I gave can't work. (I was too lazy to write down the class-based example, which *can* be made to work of course.) The question is what style of implementation you want to encourage. If you put __enter__ inside, then you encourage idempotent __exit__, which makes resource objects easier to reuse. But consider threading.RLock (a lock with the semantics of Java's monitors). Its release() is *not* idempotent, so we couldn't use the shortcut of making __enter__ and __exit__ methods of the lock itself. (I think this shortcut may be important for locks because it reduces the overhead of frequent locking -- no extra objects need to be allocated.) If you put __enter__ outside, that allows the trivial case to be written a little more simply, but also makes it hard to reuse. I skimmed your longer post about that but didn't find the conclusive evidence that this is so; all I saw was a lot of facts (you can implement scenario X in these three ways) but no conclusion. The real reason I put it inside the try was different: there's a race condition in the VM where it can raise a KeyboardInterrupt after the __enter__() call completes but before the try-suite is entered, and then __exit__() is never called. Similarly, if __enter__() is written in Python and wraps some other operation, it may complete that other operation but get a KeyboardInterrupt (or other asynchronous exception) before reaching the (explicit or implicit) return statement. Ditto for generators and yield. But I think this can be solved differently in the actual translation (as opposed to the translate-to-valid-pre-2.5-Python); the call to __exit__ can be implicit in a new opcode, and then we can at least guarantee that the interpreter doesn't check for interrupts between a successful __exit__ call and setting up the finally block. This doesn't handle an __enter__ written in Python not reaching its return; but in the presence of interrupts I don't think it's possible to write such code reliably anyway; it should be written using a with signal.blocking() around the critical code including the return. I don't want to manipulate signals directly in the VM; it's platform-specific, expensive, rarely needed, and you never know whether you aren't invoking some Python code that might do I/O, making the entire thread uninterruptible for a long time. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
At 04:57 PM 5/16/2005 +1200, Greg Ewing wrote: Guido van Rossum wrote: Also, one question: will the do protocol be added to built-in resource types? That is, locks, files, sockets, and so on? One person proposed that and it was shot down by Greg Ewing. I think it's better to require a separate wrapper. It depends on whether the resource is reusable. Why? If with is a scope statement, then it doesn't make any sense to use it with something you intend to reuse later. The statement itself is an assertion that you intend to release the resource at the end of the block, for whatever release means to that object. Releasing a file is obviously closing it, while releasing a lock is obviously unlocking it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
At 09:53 PM 5/16/2005 +1000, Nick Coghlan wrote: My PEP punts on providing a general API for passing exceptions into generators by making it an internal operation. Actually, the proposals you made almost subsume PEPs 288 and 325. All you'd need to do is: 1. move the '__del__' code to a 'close()' method and make __del__ call close() 2. make '_inject_exception()' a public API that returns the next yielded value if the exception doesn't propagate And just like that you've cleaned up the open issues from both 288 and 325, IIRC those open issues correctly. I personally think that StopIteration, TerminateIteration, KeyboardInterrupt and perhaps certain other exceptions should derive from some base class other than Exception (e.g. Raisable or some such) to help with the bare-except/except Exception problem. But that's probably best addressed by a separate PEP. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Greg Ewing wrote: Brett C. wrote: Nick's was obviously directly against looping, but, with no offense to Nick, how many other people were against it looping? It never felt like it was a screaming mass with pitchforks but more of a I don't love it, but I can deal crowd. My problem with looping was that, with it, the semantics of a block statement would be almost, but not quite, exactly like those of a for-loop, which seems to be flying in the face of TOOWTDI. And if it weren't for the can't-finalise-generators-in-a-for-loop backward compatibility problem, the difference would be even smaller. I wonder if we should reconsider PEP 340, with one change: the block iterator is required to iterate exactly once. If it iterates more than once or not at all, the interpreter raises a RuntimeError, indicating the iterator can not be used as a block template. With that change, 'break' and 'continue' will obviously affect 'for' and 'while' loops rather than block statements. Advantages of PEP 340, with this change, over PEP 343: - we reuse a protocol rather than invent a new protocol. - decorators aren't necessary. - it's a step toward more general flow control macros. At first I wasn't sure people would like the idea of requiring iterators to iterate exactly once, but I just realized the other PEPs have the same requirement. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
[Guido (responding to Fredrik Lundh's intuitive -1 on PEP 343)] Would it be better if we pulled back in the generator exit handling from PEP 340? That's a pretty self-contained thing, and would let you write try/finally around the yield. [Nick Coghlan] That would be good, in my opinion. I updated PEP 3XX to use this idea: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html With that update (to version 1.6), PEP 3XX is basically PEP 343, but injecting exceptions that occur into the template generator's internal frame instead of invoking next(). I'm in favor of the general idea, but would like to separate the error injection and finalization API for generators into a separate PEP, which would then compete with PEP 288 and PEP 325. I think the API *should* be made public if it is available internally; I don't see any implementation reasons why it would simplify the implementation if it was only available internally. Here are some issues to resolve in such a PEP. - What does _inject_exception() return? It seems that it should raise the exception that was passed into it, but I can't find this written out explicitly. - What should happen if a generator, in a finally or except clause reached from _inject_exception(), executes another yield? I'm tempted to make this a legitimate outcome (from the generator's perspective) and reserve it for some future statement that implements the looping semantics of PEP 340; the *_template decorator's wrapper class however should consider it an error, just like other protocol mismatches like not yielding the first time or yielding more than once in response to next(). Nick's code in fact does all this right, Ijust think it should be made explicit. - TerminateIteration is a lousy name, since terminate means about the same as stop, so there could be legitimate confusion with StopIteration. In PEP 340 I used StopIteration for this purpose, but someone explained this was a poor choice since existing generators may contain code that traps StopIteration for other purposes. Perhaps we could use SystemExit for this purpose? Pretty much everybody is supposed to let this one pass through since its purpose is only to allow cleanup upon program (or thread) exit. - I really don't like reusing __del__ as the method name for any kind of destructor; __del__ has all sorts of special semantics (the GC treats objects with a __del__ method specially). - The all_lines() example jars me. Somehow it bugs me that its caller has to remember to care about finalizing it. Maybe it's just not a good example; I don't see what this offers over just writing the obviously correct: for fn in filenames: with opening(fn) as f: for line in f: update_config(line) even if using the template saves a line. I doubt the use case comes up frequently enough to warrant abstracting it out. If I have to import the template to save a line here, I just made my program a bit less readable (since the first-time reader has to look up the definition of the imported all_lines template) and I didn't even save a line unless the template is used at least twice. (Note that Nick's PEP contains two typos here -- it says print lines where it should say print line and a bit later print f where again it should say print line.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
In article [EMAIL PROTECTED], Phillip J. Eby [EMAIL PROTECTED] wrote: ... I personally think that StopIteration, TerminateIteration, KeyboardInterrupt and perhaps certain other exceptions should derive from some base class other than Exception (e.g. Raisable or some such) to help with the bare-except/except Exception problem. But that's probably best addressed by a separate PEP. :) Yes, please!!! I am so sick of writing: except (SystemExit, KeyboardInterrupt): raise except Exception, e: # handle a real error but back to lurking on this discussion... -- Russell ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
[Ron Adam] So I was wondering if something like the following is feasible? [...] with opening(file1,m),opening(file2,m),opening(file3,m) as f1,f2,f3: # do stuff with files The 'with' (or whatever) statement would need a little more under the hood, but it might simplify handling multiple resources. This also reduces nesting in cases such as locking and opening. Both must succeed before the block executes. And if something goes wrong, the with statement knows and can handle each resource. The point is, each resource needs to be a whole unit, opening multiple files in one resource handler is probably not a good idea anyway. I'm -0 on this, if only because it complicates things a fair bit for a very minor improvement in functionality. There are also some semantic/syntactic questions: should this work only if there are explicit commas in the with-statement (so the compiler can generate code equivalent to nested with-statements) or should it allow a single expression to return a tuple of resource managers dynamically (so the run-time must check fora tuple each time)? It's always something we can add in the future, since it's guaranteed that this syntax (or a tuple value) is invalid in the curernt proposal. So I'd rather punt on this. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
On Mon, May 16, 2005 at 06:24:59PM +1200, Greg Ewing wrote: Brett C. wrote: Nick's was obviously directly against looping, but, with no offense to Nick, how many other people were against it looping? It never felt like it was a screaming mass with pitchforks but more of a I don't love it, but I can deal crowd. My problem with looping was that, with it, the semantics of a block statement would be almost, but not quite, exactly like those of a for-loop, which seems to be flying in the face of TOOWTDI. And if it weren't for the can't-finalise-generators-in-a-for-loop backward compatibility problem, the difference would be even smaller. Nodders, the looping construct seemed to work out fine as code people could use to get their heads around the idea. It was eye-gougingly bad as final solution. Forcing people to write an iterator for something that will almost never loop is as awkward as forcing everyone to write if statements as for dummy in range(1): if (somevar): do_true_stuff() break else: do_false_stuff() I still haven't gotten used to Guido's heart-attack inducing early enthusiasm for strange things followed later by a simple proclamation I like. Some day I'll learn that the sound of fingernails on the chalkboard is frequently followed by candy for the whole class. For now the initial stages still give me the shivers. -jackdied ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
At 08:21 PM 5/16/2005 -0400, Jack Diederich wrote: I still haven't gotten used to Guido's heart-attack inducing early enthusiasm for strange things followed later by a simple proclamation I like. Some day I'll learn that the sound of fingernails on the chalkboard is frequently followed by candy for the whole class. Heh. +1 for QOTW. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Guido van Rossum wrote: [Fredrik Lundh] unlike the original design, all you get from this is the ability to add try/finally blocks to your code without ever writing a try/finally-clause (neither in your code or in the block controller). that doesn't strike me as especially pythonic. Would it be better if we pulled back in the generator exit handling from PEP 340? That's a pretty self-contained thing, and would let you write try/finally around the yield. That would be good, in my opinion. I updated PEP 3XX to use this idea: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html With that update (to version 1.6), PEP 3XX is basically PEP 343, but injecting exceptions that occur into the template generator's internal frame instead of invoking next(). The rest of the PEP is then about dealing with the implications of allowing yield inside try/finally statements. The Rejected Options section tries to look at all the alternatives brought up in the various PEP 310, 340 and 343 discussions, and explain why PEP 3XX chooses the way it does. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Shane Hathaway wrote: Here is example A, a non-looping block statement using try: text = 'diamond' for fn in filenames: try opening(fn) as f: if text in f.read(): print 'I found the text in %s' % fn break That's a pretty way to write it! Would it be possible to extend the 'try' syntax in this way? It would certainly stress the fact that this construct includes exception handling. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Shane Hathaway wrote: Guido van Rossum wrote: Consider an application where you have to acquire *two* locks regularly: You really have to write it like this: Shane, you've already solved this one more elegantly: def lockBoth(): return combining(lock1.locking(), lock2.locking()) using the combining function you wrote earlier, which I assume will make it into the library. - Anders ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
On 5/14/05, Brett C. [EMAIL PROTECTED] wrote: Nick's was obviously directly against looping, but, with no offense to Nick, how many other people were against it looping? It never felt like it was a screaming mass with pitchforks but more of a I don't love it, but I can deal crowd. Agreed. That's certainly how I felt originally. There were a *lot* of nice features with PEP 340. The initial discussion had a lot of people enthusiastic about all the neat things they could do with it. That's disappeared now, in a long series of attempts to fix the looping issue. No-one is looking at PEP 343, or Nick's PEP 3XX, and saying hey, that's neat - I can do XXX with that!. This makes me feel that we've thrown out the baby with the bathwater. (Yes, I know PEP 342 is integral to many of the neat features, but I get the impression that PEP 342 is being lost - later iterations of the other two PEPs are going out of their way to avoid assuming PEP 324 is implemented...) Looping is definitely a wart. Looping may even be a real problem in some cases. There may be cases where an explicit try...finally remains better, simply to avoid an unwanted looping behaviour. But I'll live with that to get back the enthusiasm for a new feature that started all of this. Much better than the current yes, I guess that's good enough tone to the discussion. Paul. PS Guido - next time you get a neat idea like PEP 340, just code it and check it in. Then we can just badger you to fix the code, rather than using up all your time on discussion before there's an implementation :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Shane Hathaway [EMAIL PROTECTED] wrote:
Nick Coghlan wrote:
That would be good, in my opinion. I updated PEP 3XX to use this idea:
http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html
With that update (to version 1.6), PEP 3XX is basically PEP 343, but
injecting
exceptions that occur into the template generator's internal frame instead
of
invoking next().
The rest of the PEP is then about dealing with the implications of allowing
yield inside try/finally statements.
You might add to the PEP the following example, which could really
improve the process of building GUIs in Python:
class MyFrame(Frame):
def __init__(self):
with Panel():
with VerticalBoxSizer():
self.text = TextEntry()
self.ok = Button('Ok')
Indentation improves the readability of code that creates a hierarchy.
I've generally been fairly ambiguous about the entire PEP 310/340/343
issue. While resource allocation and release is quite useful, not a
whole heck of a lot of my code has to deal with it.
But after seeing this sketch of an example for its use in GUI
construction; I must have it!
+1 for:
- 'do' keyword (though 'try' being reused scratches the 'no new keyword'
itch for me, and is explicit about I won't loop)
- PEP 343
- try/finally + @do_template with generators
- Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Paul Moore wrote: On 5/14/05, Brett C. [EMAIL PROTECTED] wrote: Nick's was obviously directly against looping, but, with no offense to Nick, how many other people were against it looping? It never felt like it was a screaming mass with pitchforks but more of a I don't love it, but I can deal crowd. Agreed. That's certainly how I felt originally. Oh good. So I am not nuts. =) There were a *lot* of nice features with PEP 340. The initial discussion had a lot of people enthusiastic about all the neat things they could do with it. That's disappeared now, in a long series of attempts to fix the looping issue. No-one is looking at PEP 343, or Nick's PEP 3XX, and saying hey, that's neat - I can do XXX with that!. This makes me feel that we've thrown out the baby with the bathwater. (Yes, I know PEP 342 is integral to many of the neat features, but I get the impression that PEP 342 is being lost - later iterations of the other two PEPs are going out of their way to avoid assuming PEP 324 is implemented...) My feelings exactly. I was really happy and excited when it seemed like everyone really liked PEP 340 sans a few disagreements on looping and other things. Having a huge chunk of people get excited and liking a proposal was a nice contrast to the whole decorator debate. Looping is definitely a wart. Looping may even be a real problem in some cases. There may be cases where an explicit try...finally remains better, simply to avoid an unwanted looping behaviour. Which I think is actually fine if they do just use a try/finally if it fits the situation better. But I'll live with that to get back the enthusiasm for a new feature that started all of this. Much better than the current yes, I guess that's good enough tone to the discussion. Ditto. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
On 5/15/05, Paul Moore [EMAIL PROTECTED] wrote: There were a *lot* of nice features with PEP 340. The initial discussion had a lot of people enthusiastic about all the neat things they could do with it. That's disappeared now, in a long series of attempts to fix the looping issue. Having done the python-dev summary on this topic, I think the initial enthusiasm you were seeing included a lot of what if we did it this way? or what if we extended this further in another way? kind of stuff. When PEP 340 finally came out (around the end of the month), the more extreme ideas were discarded. So in some sense, PEP 340 was the reason for the lack of enthusiasm; with the semantics laid out, people were forced to deal with a specific implementation instead of a variety of wild suggestions. No-one is looking at PEP 343, or Nick's PEP 3XX, and saying hey, that's neat - I can do XXX with that!. This makes me feel that we've thrown out the baby with the bathwater. I'd be surprised if you can find many examples that PEP 340 can do that PEP 3XX can't. The only real looping example we had was auto_retry, and there's a reasonably simple solution to that in PEP 3XX. You're not going to see anyone saying hey that's neat - I can do XXX with that! because PEP 3XX doesn't add anything. But for 95% of the cases, it doesn't take anything away either. (Yes, I know PEP 342 is integral to many of the neat features, but I get the impression that PEP 342 is being lost - later iterations of the other two PEPs are going out of their way to avoid assuming PEP 324 is implemented...) Not very far out of their way. I split off PEP 342 from the original PEP 340, and it was ridiculously easy for two reasons: * the concepts are very orthogonal; the only thing really used from it in any of PEP 340 was the yield means yield None thing * there weren't *any* examples of using the continue EXPR syntax. PEP 342 is still lacking in this spot If you want to get people enthused about PEP 342 again (which is the right way to make sure it gets excepted), what would really help is a bunch of good examples of how it could be used. Looping is definitely a wart. Looping may even be a real problem in some cases. There may be cases where an explicit try...finally remains better, simply to avoid an unwanted looping behaviour. But I'll live with that to get back the enthusiasm for a new feature that started all of this. Much better than the current yes, I guess that's good enough tone to the discussion. I'm convinced that such a tone is inevitable after 30 days and over 700 messages on *any* topic. ;-) Ok, back to summarizing this fortnight's 380+ PEP 340 messages. ;-) STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Paul Moore wrote: Looping is definitely a wart. Looping may even be a real problem in some cases. There may be cases where an explicit try...finally remains better, simply to avoid an unwanted looping behaviour. I agree PEP 343 throws away too much that was good about PEP 340 - that's why I'm still updating PEP 3XX as the discussion continues. But is there anything PEP 340 does that PEP 3XX doesn't, other than letting you suppress exceptions? The only example the latest version of PEP 3XX drops since the original PEP 340 is auto_retry - and that suffers from the hidden control flow problem, so I doubt Guido would permit it, even *if* the new statement was once again a loop. And if the control flow in response to an exception can't be affected, it becomes even *harder* to explain how the new statement differs from a standard for loop. But I'll live with that to get back the enthusiasm for a new feature that started all of this. Much better than the current yes, I guess that's good enough tone to the discussion. I think the current tone is more due to the focus on addressing problems with all of the suggestions - that's always going to dampen enthusiasm. Every PEP 340 use case that doesn't involve suppressing exceptions (i.e. all of them except auto_retry) can be written under the current PEP 3XX using essentially *identical* generator code (the only difference is the statement_template decorator at the top of the generator definition) Pros of PEP 3XX 1.6 vs PEP 340: - control flow is easier to understand - can use inside loops without affecting break/continue - easy to write enter/exit methods directly on classes - template generators can be reused safely - iterator generator resource management is dealt with Cons of PEP 3XX 1.6 vs PEP 340: - no suppression of exceptions (see first pro, though) Of course, I may be biased :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
On Sun, 15 May 2005, Shane Hathaway wrote:
You might add to the PEP the following example, which could really
improve the process of building GUIs in Python:
class MyFrame(Frame):
def __init__(self):
with Panel():
with VerticalBoxSizer():
self.text = TextEntry()
self.ok = Button('Ok')
I don't understand how this would be implemented. Would a widget
function like 'TextEntry' set the parent of the widget according to
some global 'parent' variable? If so, how would 'Panel' know that
its parent is supposed to be the 'MyFrame' object?
-- ?!ng
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
[Nick Coghlan] http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html [Steven Bethard] there I see the semantics: VAR1 = stmt_enter() # Omit 'VAR1 =' if no 'as' clause exc = (None, None, None) try: try: BLOCK1 except: exc = sys.exc_info() finally: stmt_exit(*exc) Don't you want a raise after the exc = sys.exc_info()? I have the same question for Nick. Interestingly, assuming Nick meant that raise to be there, PEP 3XX and PEP 343 now have the same translation. In rev 1.10 I moved the __enter__ call out of the try-block again. Having it inside was insane: when __enter__ fails, it should do its own cleanup rather than expecting __exit__ to clean up after a partial __enter__. But some of the claims from PEP 3XX seem to be incorrect now: Nick claims that a with-statement can abstract an except clause, but that's not the case; an except clause causes the control flow to go forward (continue after the whole try statement) but the with-statement (with the raise added) always acts like a finally-clause, which implicitly re-raises the exception. So, in particular, in this example: with EXPR1: 1/0 print Boo the print statement is unreachable and there's nothing clever you can put in an __exit__ method to make it reachable. Just like in this case: try: 1/0 finally: BLOCK1 print Boo there's nothing that BLOCK1 can to to cause the print statement to be reached. This claim in PEP 3XX may be a remnant from a previous version; or it may be that Nick misunderstands how 'finally' works. Anyway, I think we may be really close at this point, if we can agree on an API for passing exceptions into generators and finalizing them, so that the generator can be written using a try/finally around the yield statement. Of course, it's also possible that Nick did *not* mean for the missing raise to be there. But in that case other claims from his PEP become false, so I'm assuming with Steven here. Nick? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Guido van Rossum wrote: Also, one question: will the do protocol be added to built-in resource types? That is, locks, files, sockets, and so on? One person proposed that and it was shot down by Greg Ewing. I think it's better to require a separate wrapper. It depends on whether the resource is reusable. It would be okay for locks since you can lock and unlock the same lock as many times as you want, but files and sockets can only be used once, so there has to be something else around them. ALthough if we use 'do', we might want to use wrappers anyway for readability, even if they're not semantically necessary. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
A additional comment (or 2) on my previous message before I go back to lurk mode. If the recommended use of each resource template is kept to a single resource, then each enter and exit can be considered a whole block of code that will either pass or fail. You can then simplify the previous template to just: # open-close pairing def opening(filename, mode): def openfile(): f = open(filename, mode) yield f f.close() return openfile with opening(file1,m),opening(file2,m),opening(file3,m) as f1,f2,f3: # do stuff with files The with statement will need to catch any opening errors at the start in case one of the opening resources fails. Close any opened resources, then re raise the first exception. The with statement will also need to catch any uncaught exceptions in the block, close any opened resources, then re raise the exception. And again when closing, it will need to catch any exceptions that occur until it has tried to close all open resources, then re raise the first exception. Although it's possible to have more than one exception occur, it should always raise the first most one as any secondary exceptions may just be a side effect of the first one. The programmer has the option to surround the 'with' block with a try except if he want to catch any exceptions raised. He should also be able to put try excepts before the yield, and after the yield, or in the block. (But not surrounding the yield, I think.) Of course he may cause himself more problems than not, but that should be his choice, and maybe he thought of some use. This might be an acceptable use case of try-except in the enter section. def openfile(firstfile, altfile, mode): try: f = open(firstfile, mode) except: f = open(altfile, mode) yield f f.close() return openfile This is still a single open resource (if it succeeds) so should work ok. Alternate closing could be possible, maybe retrying the close several time before raising it and letting the 'with' handle it. Cheers, _Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Fredrik Lundh wrote: try with opening(file) as f: body except IOError: deal with the error (you have to do this anyway) You don't usually want to do it right *there*, though. More likely you'll have something further up that deals with a variety of possible errors. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Robert Brewer wrote: There's a typo in the code snippets at the moment. The translation of the above statement is: abc = EXPR exc = () # Or (None, None, None) ? try: try: VAR = abc.__enter__() BLOCK except: exc = sys.exc_info() raise finally: abc.__exit__(exc) I think you meant abc.__exit__(*exc). Assuming that, then exc = (None, None, None) makes the most sense. If exc_info() is going to be passed as a single arg, then I'd rather have the default exc = (), so I can simply check if exc: in the __exit__ method. Also, the call to __enter__() needs to be before the try/finally block (as it is in PEP 310). Otherwise we get the releasing a lock you failed to acquire problem. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Guido van Rossum wrote: I've written up the specs for my PEP 340 redux proposal as a separate PEP, PEP 343. http://python.org/peps/pep-0343.html Those who have been following the thread Merging PEP 310 and PEP 340-redux? will recognize my proposal in that thread, which received mostly positive responses there. Please review and ask for clarifications of anything that's unclear. On the keyword front, the two keyword choices affect the naming conventions of templates differently, and I think need to be considered in that light. The naming convention for 'do' is shown in the current PEP 343. The issue I've noticed with it is that *functions* read well, but methods don't because things get out of sequence. That is, do locking(the_lock) reads well, but do the_lock.locking() does not. Whereas, using 'with', it can be written either way, and still read reasonably well (with locked(the_lock), with the_lock.locked()). The 'with' keyword also reads better if objects natively support use in 'with' blocks (with the_lock, with the_file). Guido's concern regarding file objects being reused inappropriately can be dealt with in the file __enter__ method: def __enter__(self): if self.closed: raise RuntimeError, Cannot reopen closed file handle Template generators have the exact same problem with reusability - the solution used there is raising a RuntimeError when __enter__() is called inappropriately. This would make sense as a standard idiom - if a statement template can't be reused, attempting to do so should trigger a RuntimeError the second time __enter__() is invoked. For files, it may then become the common practice to keep pathnames around, rather than open file handles. When you actually needed access to the file, the existing open builtin would suffice: with open(filename, rb) as f: for line in f: print line I've written out the PEP 343 examples below, assuming types acquire native with statement support (including Decimal contexts - I also give PEP 343 style code for Decimal contexts). PEP343 examples: 'with' keyword, native support in objects 1. A template for ensuring that a lock, acquired at the start of a block, is released when the block is left: # New methods on lock objects def __enter__(self): self.acquire() def __exit__(self, *args): self.release() Used as follows: with myLock: # Code here executes with myLock held. The lock is # guaranteed to be released when the block is left (even # if via return or by an uncaught exception). 2. A template for opening a file that ensures the file is closed when the block is left: # New methods on file objects def __enter__(self): if self.closed: raise RuntimeError, Cannot reopen closed file handle def __exit__(self, *args): self.close() Used as follows: with open(/etc/passwd) as f: for line in f: print line.rstrip() 3. A template for committing or rolling back a database transaction; this is written as a class rather than as a decorator since it requires access to the exception information: class transaction: def __init__(self, db): self.db = db def __enter__(self): self.db.begin() def __exit__(self, *args): if args and args[0] is not None: self.db.rollback() else: self.db.commit() Used as follows: with transaction(db): # Exceptions in this code cause a rollback 5. Redirect stdout temporarily: @with_template def redirected_stdout(new_stdout): save_stdout = sys.stdout sys.stdout = new_stdout yield None sys.stdout = save_stdout Used as follows: with open(filename, w) as f: with redirected_stdout(f): print Hello world This isn't thread-safe, of course, but neither is doing this same dance manually. In a single-threaded program (e.g., a script) it is a totally fine way of doing things. 6. A variant on opening() that also returns an error condition: @with_template def open_w_error(filename, mode=r): try: f = open(filename, mode) except IOError, err: yield None, err else: yield f, None f.close() Used as follows: with open_w_error(/etc/passwd, a) as f, err: if err: print IOError:, err else: f.write(guido::0:0::/:/bin/sh\n) 7.
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Guido van Rossum wrote: I've written up the specs for my PEP 340 redux proposal as a separate PEP, PEP 343. http://python.org/peps/pep-0343.html Those who have been following the thread Merging PEP 310 and PEP 340-redux? will recognize my proposal in that thread, which received mostly positive responses there. Please review and ask for clarifications of anything that's unclear. intuitively, I'm -1 on this proposal. unlike the original design, all you get from this is the ability to add try/finally blocks to your code without ever writing a try/finally-clause (neither in your code or in the block controller). that doesn't strike me as especially pythonic. (neither does the argument that just because you can use a mechanism to write inscrutable code, such a mechanism must not be made available feel right; Python's design has always been about careful tradeoffs between policy and mechanism, but this is too much policy for my taste. the original PEP 340 might have been too clever, but this reduced version feels pretty pointless). /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Fredrik Lundh wrote: unlike the original design, all you get from this is the ability to add try/finally blocks to your code without ever writing a try/finally-clause (neither in your code or in the block controller). that doesn't strike me as especially pythonic. I think the key benefit relates to the fact that correctly written resource management code currently has to be split it into two pieces - the first piece before the try block (e.g. 'lock.acquire()', 'f = open()'), and the latter in the finally clause (e.g. 'lock.release()', 'f.close()'). PEP 343 (like PEP 310 before it) makes it possible to define the correct resource management *once*, and then invoke it via a 'with' (or 'do') statement. Instead of having to check for is this file closed properly?, as soon as you write or see with open(filename) as f:, you *know* that that file is going to be closed correctly. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
On 5/14/05, Fredrik Lundh [EMAIL PROTECTED] wrote: Nick Coghlan wrote: PEP 343 (like PEP 310 before it) makes it possible to define the correct resource management *once*, and then invoke it via a 'with' (or 'do') statement. This is probably the main point for me - encapsulate the try...finally dance in such a way that the two parts are not separated by an (arbitrarily long) chunk of code. I hated the equivalent dances in C (usually malloc/free stuff in that case) and it feels awkward that this is the one real ugliness of C that Python hasn't fixed for me :-) sure, but even if you look at both the application code *and* the resource management, there are no clues that the with statement is really just a masked try/finally statement. just look at the generator example: acquire yield release what in this snippet tells you that the release part will run even if the external block raises an exception? I agree with this point, though. What I liked about the original PEP 340 was the fact that the generator was a template, with yield acting as a put the block here placeholder. but I still think that something closer to the original PEP 340 is a lot more useful. Overall, I'd agree. I'm still fond of the original PEP 340 in all its glory - the looping issue was a wart, but PEP 343 seems so stripped down as to be something entirely different, not just a fix to the looping issue. My view - PEP 343 get a +1 in preference to PEP 310. I'd like to see PEP 342 - that gets a +1 from me. Covering both these areas at once, PEP 340 would still probably be my preference, though. (I'm not convinced there's much chance of resurrecting it, though). Even it its limited form, I prefer PEP 343 to the status quo, though. Oh, and I'm leaning towards with as a keyword again, as a result of the works better with member functions argument. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Paul Moore wrote: On 5/14/05, Fredrik Lundh [EMAIL PROTECTED] wrote: Nick Coghlan wrote: PEP 343 (like PEP 310 before it) makes it possible to define the correct resource management *once*, and then invoke it via a 'with' (or 'do') statement. This is probably the main point for me - encapsulate the try...finally dance in such a way that the two parts are not separated by an (arbitrarily long) chunk of code. I hated the equivalent dances in C (usually malloc/free stuff in that case) and it feels awkward that this is the one real ugliness of C that Python hasn't fixed for me :-) sure, but even if you look at both the application code *and* the resource management, there are no clues that the with statement is really just a masked try/finally statement. just look at the generator example: acquire yield release what in this snippet tells you that the release part will run even if the external block raises an exception? I agree with this point, though. What I liked about the original PEP 340 was the fact that the generator was a template, with yield acting as a put the block here placeholder. I also think the generator as statement template works much better if the __exit__ method is able to inject the exception into the generator frame, rather than always calling next(). Maybe PEP 343 should drop any suggestion of using generators to define these things, and focus on the PEP 310 style templates. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
At 01:55 PM 5/14/2005 +0200, Fredrik Lundh wrote: also, come to think of it, adding a new statement just to hide try/finally statements is a waste of statement space. why not just enhance the existing try statement? let try with opening(file) as f: body except IOError: deal with the error (you have to do this anyway) behave like try: f = opening(file) try: try: body except: exc = sys.exc_info() else: exc = None finally: f.__cleanup__(exc) except IOError: deal with the error and you're done. (or you could use __enter__ and __exit__ instead, which would give you a variation of PEP-343-as-I-understand-it) I like this, if you take out the with part, change the method names to __try__ and __finally__, and allow try to work as a block on its own if you've specified an expression. i.e.: try opening(filename) as f: # do stuff try locking(self.__lock): # do stuff try redirecting_stdout(f): # something try decimal.Context(precision=23): # okay, this one's a little weird try self.__lock: # and so's this; nouns read better with a gerund wrapper and I'd make the translation be: try: __exc = () VAR = EXPR.__try__() try: try: BODY except: __exc = sys.exc_info() raise finally: EXPR.__finally__() # except/else/finally clauses here, if there were any in the original try but I still think that something closer to the original PEP 340 is a lot more useful. I agree, but mainly because I'd like to be able to allow try/finally around yield in generators, be able to pass exceptions into generators, and tell generators to release their resources. :) I do think that the PEP 340 template concept is much more elegant than the various PEP 310-derived approaches, though. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
[Fredrik Lundh] intuitively, I'm -1 on this proposal. So we need to do better. Do you just prefer all of PEP 340? What about the objections against it? The mostly unnecessary loopiness in particular? unlike the original design, all you get from this is the ability to add try/finally blocks to your code without ever writing a try/finally-clause (neither in your code or in the block controller). that doesn't strike me as especially pythonic. Would it be better if we pulled back in the generator exit handling from PEP 340? That's a pretty self-contained thing, and would let you write try/finally around the yield. (neither does the argument that just because you can use a mechanism to write inscrutable code, such a mechanism must not be made available feel right; Python's design has always been about careful tradeoffs between policy and mechanism, but this is too much policy for my taste. the original PEP 340 might have been too clever, but this reduced version feels pretty pointless). Maybe. It still solves the majority of use cases for PEP 340, most of which are try/finally abstractions. Maybe I'm overreacting to Raymond Chen's rant about flow-control macros -- but having had to maintain code once that was riddled with these, it rang very true. PEP 340 is still my favorite, but it seems there's too much opposition to it, so I'm trying to explore alternatives; at the same time I *really* dislike the complexities of some of the non-looping counterproposals (e.g. Nick Coghlan's PEP 3XX or the proposals that make every keyword associated with 'try' a method). -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Guido van Rossum wrote: [Fredrik Lundh] intuitively, I'm -1 on this proposal. Just to toss in my opinion, I prefer PEP 340 over 343 as well, but not so much to give 343 a -1 from me. [SNIP - question of how to handle argument against 340 being a loop which I never totally got since you know it ahead of time so you just deal with it] [SNIP] Maybe I'm overreacting to Raymond Chen's rant about flow-control macros -- but having had to maintain code once that was riddled with these, it rang very true. You might be overreacting. I read the essay and I understand his arguments having been taught how to program in Scheme. But I don't think this is as serious. People will have the semantics of the block statement explained to them so how that works will be clear. And at that point they just have to track down the generator or iterator that the block statement is using. If you think about it, how is it different than the implicit iter() call on a 'for' loop along with the implicit next() call each time through the loop? Just because there is an implicit closing call back into the block generator at the end of a block statement? Doesn't seem so bad to me or that much of a stretch from the magic of a 'for' loop to be that huge of a thing. I think Raymond was reeling against arbitrary macro creation that hides flow control. We don't have that here. What we have is a clearly defined statement that does some very handy syntactic sugar for us. It doesn't feel as arbitrary as what Lisp and Scheme allow you to do. PEP 340 is still my favorite, but it seems there's too much opposition to it, so I'm trying to explore alternatives; at the same time I *really* dislike the complexities of some of the non-looping counterproposals (e.g. Nick Coghlan's PEP 3XX or the proposals that make every keyword associated with 'try' a method). Nick's was obviously directly against looping, but, with no offense to Nick, how many other people were against it looping? It never felt like it was a screaming mass with pitchforks but more of a I don't love it, but I can deal crowd. And as for the overly complex examples, that I believed stemmed from people realizing what might be possible if the statement was extended and tweaked this way or that so as to be able to do just one more thing. But that happens with every proposal; seemed like standard feature creep. The reason we have a BDFL is to tell us that yes, we could get the jumbo sized candy bar for $2 more but there is no way you will be able to finish that much chocolate before it melts all over your hands and get it all over your nice PyCon t-shirt. But then again I don't know if you got private emails asking to see if PEP 340 weighed as much as wood so it could be burned at the stake for being a witch. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Guido van Rossum wrote:
[Nick Coghlan]
Also, the call to __enter__() needs to be before the try/finally block (as it
is
in PEP 310). Otherwise we get the releasing a lock you failed to acquire
problem.
I did that on purpose. There's a separate object ('abc' in the
pseudo-code of the translation) whose __enter__ and __exit__ methods
are called, and in __enter__ it can keep track of the reversible
actions it has taken.
Consider an application where you have to acquire *two* locks regularly:
def lockBoth():
got1 = got2 = False
lock1.acquire(); got1 = True
lock2.acquire(); got2 = True
yield None
if got2: lock2.release()
if got1: lock1.release()
If this gets interrupted after locking lock1 but before locking lock2,
it still has some cleanup to do.
That code is incorrect, though. Say lockBoth() acquires lock1 but then
lock2.acquire() throws an exception. (Maybe the lock requires some I/O
operation, and the operation fails.) The interpreter will never reach
the yield statement and lock1 will never be released.
You really have to write it like this:
def lockBoth():
lock1.acquire()
try:
lock2.acquire()
except:
lock1.release()
raise
yield None
try:
lock2.release()
finally:
lock1.release()
I know that this complicates simpler use cases, and I'm not 100% sure
this is the right solution; but I don't know how else to handle this
use case.
If __enter__ raises an exception, it has to clean up after itself before
propagating the exception. __exit__ shouldn't be called if __enter__ fails.
Shane
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Brett C. wrote: Guido van Rossum wrote: PEP 340 is still my favorite, but it seems there's too much opposition to it, so I'm trying to explore alternatives; at the same time I *really* dislike the complexities of some of the non-looping counterproposals (e.g. Nick Coghlan's PEP 3XX or the proposals that make every keyword associated with 'try' a method). Nick's was obviously directly against looping, but, with no offense to Nick, how many other people were against it looping? It never felt like it was a screaming mass with pitchforks but more of a I don't love it, but I can deal crowd. PEP 340 is very nice, but it became less appealing to me when I saw what it would do to break and continue statements. text = 'diamond' for fn in filenames: opening(fn) as f: if text in f.read(): print 'I found the text in %s' % fn break I think it would be pretty surprising if the break didn't stop the loop. Here's a new suggestion for PEP 340: use one keyword to start a block you don't want to loop, and a different keyword to start a block that can loop. If you specify the non-looping keyword but the block template produces more than one result, a RuntimeError results. Here is example A, a non-looping block statement using try: text = 'diamond' for fn in filenames: try opening(fn) as f: if text in f.read(): print 'I found the text in %s' % fn break In example A, the break statement breaks the for loop. If the opening() iterator returns more than one result, a RuntimeError will be generated by the Python interpreter. Here is example B, a looping block statement using in, adapted from PEP 340: in auto_retry(3, IOError) as attempt: f = urllib.urlopen(http://python.org/peps/pep-0340.html;) print f.read() Note that I introduced no new keywords except as, and the syntax in both cases is currently illegal. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Guido van Rossum wrote:
[Nick Coghlan]
Also, the call to __enter__() needs to be before the try/finally block (as it
is
in PEP 310). Otherwise we get the releasing a lock you failed to acquire
problem.
I did that on purpose. There's a separate object ('abc' in the
pseudo-code of the translation) whose __enter__ and __exit__ methods
are called, and in __enter__ it can keep track of the reversible
actions it has taken.
Consider an application where you have to acquire *two* locks regularly:
def lockBoth():
got1 = got2 = False
lock1.acquire(); got1 = True
lock2.acquire(); got2 = True
yield None
if got2: lock2.release()
if got1: lock1.release()
If this gets interrupted after locking lock1 but before locking lock2,
it still has some cleanup to do.
I know that this complicates simpler use cases, and I'm not 100% sure
this is the right solution; but I don't know how else to handle this
use case.
If we retained the ability to inject exceptions into generators, this would be
written with the extremely natural:
@with template:
def lockboth():
lock1.acquire()
try:
lock2.acquire()
try:
yield
finally:
lock2.release()
finally:
lock1.release()
Or, even more simply:
@with_template:
def lockboth():
with lock1:
with lock2:
yield
I think Fredrik's intuition is on to something - PEP 343 has scaled the idea
back *too* far by throwing away the injection of exceptions into generator
templates, when the only major objection was to the looping nature of the
proposal.
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Guido van Rossum wrote:
I've written up the specs for my PEP 340 redux proposal as a
separate PEP, PEP 343.
http://python.org/peps/pep-0343.html
Those who have been following the thread Merging PEP 310 and PEP
340-redux? will recognize my proposal in that thread, which received
mostly positive responses there.
Please review and ask for clarifications of anything that's unclear.
+1 here.
The stdout redirection example needs to be corrected to avoid yielding inside a
try/finally though:
5. Redirect stdout temporarily:
@do_template
def redirecting_stdout(new_stdout):
save_stdout = sys.stdout
try:
sys.stdout = new_stdout
except:
sys.stdout = save_stdout
raise
else:
yield None
sys.stdout = save_stdout
Used as follows:
do opening(filename, w) as f:
do redirecting_stdout(f):
print Hello world
This could be left as the more elegant original if iterator finalisation (e.g.
using a __finish__() slot) came in at the same time as user defined
statements, allowing the above to be written naturally with try/finally.
Arnold deVos's HTML tagging example would need access to the exception
information and could be rewritten as a class:
def tag(object):
def __init__(self, name):
self.name = cgi.escape(name)
def __enter__(self):
print '%s' % self.name
return self.name
def __exit__(self, *exc_info):
if not exc_info or exc_info[0] is None:
print '/%s' % self.name
Used as follows::
do tag('html'):
do tag('head'):
do tag('title'):
print 'A web page'
do tag('body'):
for par in pars:
do tag('p'):
print par
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
[Phillip J. Eby] May I suggest this alternative translation in the Specification section: abc = EXPR __args = () # pseudo-variable, not visible to the user try: VAR = abc.__enter__() try: BLOCK except: __args = sys.exc_info() finally: abc.__exit__(*__args) Done (except you forgot to add a raise to the except claise). While slighly more complex than the current translation, the current translation seems a bit misleading to me. OTOH, that may simply be because I see the *sys.exc_info() part and immediately wonder what happens when there was no exception, and sys.exc_info() contains some arbitrary previous data... Right. Well, anyway, the actual implementation will just get the exception info from the try/finally infrastructure -- it's squirreled away somewhere on the stack even if sys.exc_info() (intentionally) doesn't have access to it. Also, one question: will the do protocol be added to built-in resource types? That is, locks, files, sockets, and so on? One person proposed that and it was shot down by Greg Ewing. I think it's better to require a separate wrapper. Or will there instead be macros like the opening and locking templates? I notice that grammatically, do gerund works a lot better than do noun, so all of your examples are words like locking, blocking, opening, redirecting, and so on. This makes it seem awkward for e.g. do self.__lock, which doesn't make any sense. But the extra call needed to make it do locking(self.__lock) seems sort of gratuitous. Maybe. There seems to be a surge of proponents for 'do' at the moment. It makes me wonder if with or using or some similar word that works better with nouns might be more appropriate, as then it would let us just add the resource protocol to common objects, and still read well. For example, a Decimal Context object might implement __enter__ by setting itself as the thread-local context, and __exit__ by restoring the previous context.do aDecimalContext doesn't make much sense, but with aDecimalContext or using aDecimalContext reads quite nicely. Maybe. I think we ought to implement the basic mechanism first and then decide how fancy we want to get, so I'd rather not get into this in the PEP. I'll add this to the PEP. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
[Nick Coghlan] The stdout redirection example needs to be corrected to avoid yielding inside a try/finally though: Thanks -- fixed now. This could be left as the more elegant original if iterator finalisation (e.g. using a __finish__() slot) came in at the same time as user defined statements, allowing the above to be written naturally with try/finally. Let's not try to tie this to other features. I tried that with PEP 340 and you know the mess it became. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - Abstract Block Redux
Guido van Rossum wrote: I've written up the specs for my PEP 340 redux proposal as a separate PEP, PEP 343. http://python.org/peps/pep-0343.html Those who have been following the thread Merging PEP 310 and PEP 340-redux? will recognize my proposal in that thread, which received mostly positive responses there. Please review and ask for clarifications of anything that's unclear. There's a typo in the code snippets at the moment. The translation of the above statement is: abc = EXPR exc = () # Or (None, None, None) ? try: try: VAR = abc.__enter__() BLOCK except: exc = sys.exc_info() raise finally: abc.__exit__(exc) I think you meant abc.__exit__(*exc). Assuming that, then exc = (None, None, None) makes the most sense. If exc_info() is going to be passed as a single arg, then I'd rather have the default exc = (), so I can simply check if exc: in the __exit__ method. Robert Brewer System Architect Amor Ministries [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
