Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: Hmm, with that approach, a code inspection tool like pychecker could be used to pick up the slack, and flag generators which have a yield inside a try/finally or a user defined statement without applying the needs finalisation decorator What about giving them an __exit__ method if and only if they have a yield inside a try/finally? Old generators won't be doing that, because it's currently illegal. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Greg Ewing wrote: Nick Coghlan wrote: Hmm, with that approach, a code inspection tool like pychecker could be used to pick up the slack, and flag generators which have a yield inside a try/finally or a user defined statement without applying the needs finalisation decorator What about giving them an __exit__ method if and only if they have a yield inside a try/finally? Old generators won't be doing that, because it's currently illegal. It's possible to create a generator that does not contain a finally, but still needs cleanup. def gen(): try: yield except: print 'cleanup' raise Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam wrote: There seems to be some confusion as to weather or not 'for's will do finalizing. So I was trying to stress I think regular 'for' loops should not finalize. They should probably give an error if an object with an try-finally in them or an __exit__ method. But if the for-loop can tell whether the iterator needs finalizing or not, why not have it finalize the ones that need it and not finalize the ones that don't? That would be backwards compatible, since old for-loops working on old iterators would work as before. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam [EMAIL PROTECTED] wrote: The argument over whether blocks should loop, I believe has been had; they should. The various use cases involve multi-part transactions and such. I think so now too, I had thought as Nick does earlier this week that the non-looping version was cleaner, but changed my mind when I realized that looping blocks could be made to work for those in a simple and understandable way. I wasn't expressing my opinion, I was attempting to express as to where the discussion went and concluded. I honestly can't remember having an opinion on the subject, but I seem to have convinced Nick earlier that they shouldn't loop, and he (re-)convinced me that indeed, they shouldn't loop. I think maybe another alternative is a break buffer or cue. Where you push a 'break' onto the buffer and then execute a 'break' to break the current loop, The 'break' in the buffer then breaks the next loop out as soon as the current loop exits, etc. [snip] It needs some polish I think. ;-) Goodness, the horror! When implementation details start bleeding their way into actual language constructs (using a continue/break stack in order to control the flow of nested loops), that's a good clue that an idea has gone a bit too far. I would honestly prefer gotos, and I would prefer having no change to existing syntax to gaining gotos. It's kind of funny. Every month I spend in python-dev, I feel less inclined to want to change the Python language (except for the relative import I need to finish implementing). Not because it is a pain in the tookus (though it is), but because many times it is my immediate sense of aesthetics that causes me to desire change, and my future of code maintenance makes me think forward to understanding Python 2.3 in the context of Python 2.9 . - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Greg Ewing wrote: Ron Adam wrote: There seems to be some confusion as to weather or not 'for's will do finalizing. So I was trying to stress I think regular 'for' loops should not finalize. They should probably give an error if an object with an try-finally in them or an __exit__ method. But if the for-loop can tell whether the iterator needs finalizing or not, why not have it finalize the ones that need it and not finalize the ones that don't? That would be backwards compatible, since old for-loops working on old iterators would work as before. That's why I suggested to have the behaviour depend on what is passed in as EXPR. for VAR in EXPR: BLOCK could be translated to: __cleanup = False __itr = EXPR if not isinstance(__itr,iterator): __itr = iter(__itr) __cleanup = True while True: try: VAR = __itr.next() except StopIteration: break BLOCK if __cleanup: __itr.__exit__() Which would require isinstance(__itr,iterator) or equivalent to act as a robust test on iterators. I'll leave 'for' with an 'else' clause as an exercise to the reader. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Jim Jewett [EMAIL PROTECTED] wrote: Nick Coghlan ncoghlan at gmail.com wrote: Josiah Carlson wrote: This has the benefit that an arbitrary block of code can be named, and a named TerminateBlock used to exit it. ... I suspect any such implementation is going to need to use exceptions for the guts of the flow control, even if that use isn't visible to the programmer. Not necessarily. If I were implementing such a thing; any time arbitrary break/continues (to a loop that isn't the deepest) were used in nested loops, I would increment a counter any time a loop was entered, and decrement the counter any time a loop was exited. ... When named blocks are used in Lisp, they often cross function boundaries. Given that, the number of intervening loops could change depending on external variables. Since you would have to pop frames anyhow, Exceptions are the right way to do it. I wasn't talking about cross-function blocks/named blocks. I was strictly talking about nested loops as they currently exist in Python. If you limited the named-block gotos to within a single function/method, then the loop counter would work (and you could limit obfuscation). Unfortunately, you would lose most of the power of named blocks, while still paying the full ugliness price. That's fine, I don't want named loops or blocks anyhow. I was merely offering an implementation that did not require exceptions, and was necessarily fast (proving both that it could be fast and not require exceptions). You would also encourage people to inline things that ought to be separate functions. I wouldn't go that far. If one were to introduce such functionality, it would be to ease control flow within nested for/while/blocks. Whether or not that lead to people inlining code, who are we to say? It would, however, complicate the 'inline function' decorator that I seem to have lost my link to. In case it isn't clear, I think named loops would be a mistake. I wanted them when I first started, but ... at the moment, I can't think of any usage that wasn't an ugly speed hack, which is at least more explicit with the raise Found idiom. Don't get me wrong, I think they would be a mistake as well, but they would solve the 'does a break statement in a block break its enclosing loop' question, as well as general nested loop flow control issues. Now that we both agree that they shouldn't be done, maybe one of us should write a PEP for Guido to rule on so that we never have to hear about loop naming (heh). - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam [EMAIL PROTECTED] wrote: Josiah Carlson wrote: Ron Adam [EMAIL PROTECTED] wrote: There's also the possibility to use conditional looping based on the value returned from the generator. do VAR from EXPR if VAR==CONST: BLOCK This is a bit verbose, but it reads well. :-) Reading well or not, this is not really an option for the same reasons why... for VAR in EXPR1 if EXPR2: or for VAR in EXPR1 while EXPR2: are not options. Keep it simple. Yes, so just do [VAR from] EXPR1: Regardless of the 'finalization' syntax, I'm talking about the fact that including extra 'if EXPR' or 'while EXPR' is not going to be an option. 3. Do-loops: An generator based loop with finalization: This could be both single and multiple pass. The difference is determined by weather or not the generator used loops the yield statement or not. Offering only generator-based finalization loops is, as I understand it, not an option. It could also include classes with __exit__ methods which are really just more complex generators when used this way. But isn't this what PEP340 *already* proposes? Or am I missing a subtle distinction here. It is, in fact, what PEP 340 already proposes. Let us take a step back for a moment and realize that this entire discussion is going around in circles. From what I understand, we all agree: 1. There should be some mechanism X which signals that an indented suite is a 'block statement'. Such blocks are described and finalized as per PEP 340 (or whatever derivative gets accepted). 2. Standard for/while loops should not be finalized in a timely fashion, because testing for the proper methods would necessarily slow down large amounts of current Python, so should be left to the garbage collector. 3. A note as to the additional overhead of finalized blocks should be mentioned in the source code of the finalization implementation, and a note on their performace characteristics may or may not need to be listed in the language reference. What there is still discussion over: 4. What the syntax should be. 5. Whether or not it loops. 6. Method names. I find the answers to 4,5,6 to be a matter of opinion, and like many, I have my own. However, I do not feel strongly enough about 4,5,6 to argue about my opinion (I've been attempting to re-state where the conversation went for the last 2 weeks, as I have managed to read each and every email about the subject at hand *ick*). - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: Ron Adam [EMAIL PROTECTED] wrote: Yes, so just do [VAR from] EXPR1: Regardless of the 'finalization' syntax, I'm talking about the fact that including extra 'if EXPR' or 'while EXPR' is not going to be an option. Yes, I meant for the syntax to be the shorter form, not for the programmer to just leave off the end. But isn't this what PEP340 *already* proposes? Or am I missing a subtle distinction here. It is, in fact, what PEP 340 already proposes. Let us take a step back for a moment and realize that this entire discussion is going around in circles. I think so too. From what I understand, we all agree: 1. There should be some mechanism X which signals that an indented suite is a 'block statement'. Such blocks are described and finalized as per PEP 340 (or whatever derivative gets accepted). +1 2. Standard for/while loops should not be finalized in a timely fashion, because testing for the proper methods would necessarily slow down large amounts of current Python, so should be left to the garbage collector. +1 Also add to this, it is not always desirable to finalize an object after use in a for loop. 3. A note as to the additional overhead of finalized blocks should be mentioned in the source code of the finalization implementation, and a note on their performace characteristics may or may not need to be listed in the language reference. +1 What there is still discussion over: 4. What the syntax should be. 5. Whether or not it loops. 6. Method names. I find the answers to 4,5,6 to be a matter of opinion, and like many, I have my own. However, I do not feel strongly enough about 4,5,6 to argue about my opinion (I've been attempting to re-state where the conversation went for the last 2 weeks, as I have managed to read each and every email about the subject at hand *ick*). - Josiah I think you clarified this well. Item 4: A list of possible syntax's with a vote at some point should do. Item 5: (A.) More use case's for looping blocks need to be found. I think there may be some or many that are not obvious at the moment. (B.) It may not cost much in performance to include the looping behavior. Maybe this should be put off till there is a working version of each, then comparisons of performance can be made in different situations? _Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Raymond Hettinger wrote: [Nick Coghlan] The number of good use cases for a looping block statement currently stands at exactly 1 (auto_retry). Every other use case suggested (locking, opening, suppressing, etc) involves factoring out try statement boiler plate that is far easier to comprehend with a single pass user defined statement. I would like to offer up one additional use case, eliminating redundant code in a do-while construct: def do_while(cond_func): yield while cond_func(): yield block do_while(lambda: ab): sequence of actions affecting a and b Nice example, but it doesn't need to intercept exceptions the way auto_retry does. Accordingly, a 'for' loop which makes [VAR in] optional would do the job just fine: for do_while(lambda: ab): do it Even today, the following works: def do_while(cond_func): yield None while cond_func(): yield None for _ in do_while(lambda: ab): do it Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam wrote: That's sort of why I'm against changing 'for', and for adding the new loop/block. I see it as a loop with a higher level of abstraction. A new tool to be used in new ways, but I want to keep my old dependable tools too. But if there's too much overlap in functionality between the old and the new tool, you're in danger of losing TOOWTDI. Random thought for the day: Programming tools are different from physical tools. I own quite a few different screwdrivers, several of which would be more or less equally good for any particular screw, and this isn't a problem. But I don't have a big crowd of people looking over my shoulder while I work, all trying to figure out why I chose one particular screwdriver over another, and decide which would be the best screwdriver to use on their screws. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: 2. Standard for/while loops should not be finalized in a timely fashion, because testing for the proper methods would necessarily slow down large amounts of current Python, so should be left to the garbage collector. I'm not convinced about that. If implemented properly, it could be as simple as testing whether a slot of a type object is populated during processing of the bytecode which causes exit from the loop. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: Ron Adam wrote: I agree, re-using or extending 'for' doesn't seem like a good idea to me. I agree that re-using a straight 'for' loop is out, due to performance and compatibility issues with applying finalisation semantics to all such iterative loops (there's a reason the PEP redraft doesn't suggest this). However, it makes sense to me that a for loop with finalisation should actually *be* a 'for' loop - just with some extra syntax to indicate that the iterator is finalised at the end of the loop. Question: Is the 'for' in your case iterating over a sequence? or is it testing for an assignment to determine if it should continue? The difference is slight I admit, and both views can be said to be true for 'for' loops iterating over lists also. But maybe looking at it as a truth test of getting something instead of an iteration over a sequence would fit better? When a variable to assign is not supplied then the test would be of a private continue-stop variable in the iterator or a StopIteration exception. However, as you say, 'del' isn't great for the purpose, but I was trying to avoid introduding yet another keyword. I didn't say, that was Josiah, but I agree 'del' is not good. An obvious alternative is to use 'finally' instead: for [finally] [VAR in] EXPR: BLOCK1 else: BLOCK2 It still doesn't read all that well, but at least the word more accurately reflects the semantics involved. How about: keyword? [VAR from] EXPR: Could 'from' be reused in this context? If the keyword chosen is completely different from 'for' or 'while', then it doesn't need a 'del' or 'finally' as that can be part of the new definition of whatever keyword is chosen. I suggested reusing 'while' a few days ago because it fit the situation well, but come to the conclusion reusing either 'for' or 'while' should both be avoided. So you might consider 'do', Guido responded with the following the other day: #quote [Greg Ewing] How about 'do'? do opening(filename) as f: ... do locking(obj): ... do carefully(): # :-) ... I've been thinking of that too. It's short, and in a nostalgic way conveys that it's a loop, without making it too obvious. (Those too young to get that should Google for do-loop. :-) I wonder how many folks call their action methods do() though. #endquote So it's not been ruled out, or followed though with, as far as I know. And I think it will work for both looping and non looping situations. The last option is to leave finalisation out of the 'for' loop syntax, and introduce a user defined statement to handle the finalisation: Yes, leaving it out of 'for' loop syntax is good. I don't have an opinion on user defined statements yet. But I think they would be somewhat slower than a built in block that does the same thing. Performance will be an issue because these things will be nested and possibly quite deeply. I wonder how much effect adding, 'for-next' and the 'StopIteration' exception check as proposed in PEP340, will have on 'for''s performance. I'm not sure what you mean here - 'for' loops already use a StopIteration raised by the iterator to indicate that the loop is complete. The code you posted can't work, since it also intercepts a StopIteration raised in the body of the loop. Oops, meant that to say 'for-else' above ... The 'else' is new isn't it? I was thinking that putting a try-except around the loop does the same thing as the else. Unless I misunderstand it's use. But you are right, it wouldn't work if the loop catches the StopIteration. I think a completely separate looping or non-looping construct would be better for the finalization issue, and maybe can work with class's with __exit__ as well as generators. The PEP redraft already proposes a non-looping version as a new statement. However, since generators are likely to start using the new non-looping statement, it's important to be able to ensure timely finalisation of normal iterators as well. Huh? I thought a normal iterator or generator doesn't need finalization? If it does, then it's not normal. Has a word been coined for iterators with try-finally's in them yet? Ron_Adam :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam [EMAIL PROTECTED] wrote: Josiah Carlson wrote: I think a completely separate looping or non-looping construct would be better for the finalization issue, and maybe can work with class's with __exit__ as well as generators. From what I understand, the entire conversation has always stated that class-based finalized objects and generator-based finalized objects will both work, and that any proposal that works for one, but not the other, is not sufficient. That's good to hear. There seems to be some confusion as to weather or not 'for's will do finalizing. So I was trying to stress I think regular 'for' loops should not finalize. They should probably give an error if an object with an try-finally in them or an __exit__ method. I'm not sure what the current opinion on that is. But I didn't see it in any of the PEPs. It's not a matter of 'will they be finalized', but instead a matter of 'will they be finalized in a timely manner'. From what I understand; upon garbage collection, any generator-based resource will be finalized via __exit__/next(exception)/... and any class-based resource will have its __del__ method called (as long as it is well-behaved), which can be used to call __exit__... Having it loop has the advantage of making it break out in a better behaved way. What you have just typed is nonsense. Re-type it and be explicit. It was a bit brief, sorry about that. :-) To get a non-looping block to loop, you will need to put it in a loop or put a loop in it. In the first case, doing a 'break' in the block doesn't exit the loop. so you need to add an extra test for that. In the second case, doing a 'break' in the loop does exit the block, but finishes any code after the loop. So you may need an extra case in that case. Having a block that loops can simplify these conditions, in that a break alway exits the body of the block and stops the loop. A 'continue' can be used to skip the end of the block and start the next loop early. And you still have the option to put the block in a loop or loops in the block and they will work as they do now. I hope that clarifies what I was thinking a bit better. That is the long-standing nested loops 'issue', which is not going to be solved here, nor should it be. I am not sure that any solution to the issue will be sufficient for everyone involved. The closest thing to a generic solution I can come up with would be to allow for the labeling of for/while loops, and the allowing of break/continue label, which continues to that loop (breaking all other loops currently nested within), or breaks that loop (as well as all other loops currently nested within). Perhaps something like... while ... label 'foo': for ... in ... label 'goo': block ... label 'hoo': if ...: #equivalent to continue 'hoo' continue elif ...: continue 'goo' elif ...: continue 'foo' else: break 'foo' Does this solve the nested loop problem? Yes. Do I like it? Not really; three keywords in a single for/block statement is pretty awful. On the upside, 'label' doesn't need to be a full-on keyword (it can be a partial keyword like 'as' still seems to be). Enough out of me, good night, - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: Eric Nieuwland [EMAIL PROTECTED] wrote: I don't know. Using 'del' in that place seems ackward to me. Why not use the following rule: for [VAR in] EXPR: SUITE If EXPR is an iterator, no finalisation is done. If EXPR is not an iterator, it is created at the start and destroyed at the end of the loop. You should know why that can't work. If I pass a list, is a list an iterator? No, but it should neither be created nor destroyed before or after. I suggested to create AN ITERATOR FOR THE LIST and destroy that at the end. The list itself remains untouched. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam wrote: Question: Is the 'for' in your case iterating over a sequence? or is it testing for an assignment to determine if it should continue? Iterating over a sequence. If it's single-pass (and always single pass), you should use a user defined statement instead. The difference is slight I admit, and both views can be said to be true for 'for' loops iterating over lists also. But maybe looking at it as a truth test of getting something instead of an iteration over a sequence would fit better? When a variable to assign is not supplied then the test would be of a private continue-stop variable in the iterator or a StopIteration exception. No, that's the use case for user defined statements - if __enter__ raises TerminateBlock, then the body of the statement is not executed. What the for-loop part of the redrafted PEP is about is whether or not there should be an easy way to say iterate over this iterator, and finalise it afterwards, regardless of how the iteration is ended, rather than having to use a try/finally block or a user defined statement for that purpose. I think I need to reorder those two sections - introduce user-defined statements first, then consider whether or not to add direct finalisation support to for loops. If the keyword chosen is completely different from 'for' or 'while', then it doesn't need a 'del' or 'finally' as that can be part of the new definition of whatever keyword is chosen. That's the technique suggested for the single-pass user defined statements. However, a 'for loop with finalisation' is *still fundamentally an iterative loop*, and the syntax should reflect that. So you might consider 'do', Guido responded with the following the other day: [snip quote from Guido] So it's not been ruled out, or followed though with, as far as I know. And I think it will work for both looping and non looping situations. The same keyword cannot be used for the looping vs non-looping construct, because of the effect on the semantics of break and continue statements. The non-looping construct is the more fundamental of the two, since it can replace any current try/except/else/finally boilerplate, without any concern over whether or not the contained code using break or continue statements. A looping construct alters the meanings of those statements. The last option is to leave finalisation out of the 'for' loop syntax, and introduce a user defined statement to handle the finalisation: Yes, leaving it out of 'for' loop syntax is good. I don't have an opinion on user defined statements yet. But I think they would be somewhat slower than a built in block that does the same thing. What do you mean by 'built in block'? The user defined statements of the PEP redraft are simply a non-looping version of PEP 340's anonymous block statements. Oops, meant that to say 'for-else' above ... The 'else' is new isn't it? I was thinking that putting a try-except around the loop does the same thing as the else. Unless I misunderstand it's use. No, the else clause on loops is a little known part of present day Python - it executes whenever the loop terminates naturally (i.e. not via a break statement). The only thing PEP 340 adds to for loops is the semantics to handle an argument to continue statements - it adds nothing to do with finalisation. My PEP redraft, on the other hand, suggests the introduction of a 'for loop with finalisation' that works fairly similarly to PEP 340's anonymous block statements. The PEP redraft already proposes a non-looping version as a new statement. However, since generators are likely to start using the new non-looping statement, it's important to be able to ensure timely finalisation of normal iterators as well. Huh? I thought a normal iterator or generator doesn't need finalization? If it does, then it's not normal. Has a word been coined for iterators with try-finally's in them yet? An example was posted that looked like this: def all_lines(filenames): for name in filenames: stmt opening(name) as f: for line in f: yield line This is clearly intended for use as an iterator - it returns a bunch of lines. However, if the iterator is not finalised promptly, then the file that provided the last line may be left open indefinitely. By making such an iterator easy to write, it behooves the PEP to make it easy to use correctly. This need *can* be met by the 'consuming' user defined statement I posted earlier, but a more elegant solution is to be able to iterate over this generator normally, while also being able to ask Python to ensure the generator is finalised at the end of the iteration. Regards, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
On 5/8/05, Jp Calderone [EMAIL PROTECTED] wrote: If such a construct is to be introduced, the ideal spelling would seem to be: for [VAR in] EXPR: BLOCK1 finally: BLOCK2 While I have not been following this discussion at all (I don't have the energy or time to follow the development of yet another proposal - I'll wait for the PEP) this does read more naturally to me than any of the other contortions I've seen passing by. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Paul Moore wrote: On 5/8/05, Jp Calderone [EMAIL PROTECTED] wrote: If such a construct is to be introduced, the ideal spelling would seem to be: for [VAR in] EXPR: BLOCK1 finally: BLOCK2 While I have not been following this discussion at all (I don't have the energy or time to follow the development of yet another proposal - I'll wait for the PEP) this does read more naturally to me than any of the other contortions I've seen passing by. Given this for loop syntax: for VAR in EXPR: BLOCK1 else: BLOCK2 finally: BLOCK3 And these semantics when a finally block is present: itr = iter(EXPR1) exhausted = False try: while True: try: VAR1 = itr.next() except StopIteration: exhausted = True break BLOCK1 if exhausted: BLOCK2 finally: try: BLOCK3 finally: itr_exit = getattr(itr, __exit__, None) if itr_exit is not None: try: itr.__exit__(TerminateBlock) except TerminateBlock: pass Loop on this iterator and finalise when done would be written: for item in itr: process(item) finally: pass If you just want the finally clause, without finalising the iterator, you write it as you would now: try: for item in itr: process(item) finally: finalisation() I like it - I'll update the PEP redraft to use it instead of the 'del' idea. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: The whole PEP draft can be found here: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html I've updated this based on the feedback so far. The biggest change is that I've dropped the 'del' idea in favour of an optional 'finally' clause on for loops that finalises the iterator in addition to executing the code contained in the clause. I also added additional description of the purpose of user defined statements (factoring out exception handling boilerplate that is not easily factored into a separate function), and fixed the semantics so that __exit__() is called without an argument when the statement exits cleanly (previously, a template could not tell if the statement exited cleanly or not). I expanded on the generator section, indicating that the __exit__ method simply invokes next() if no exception is passed in (this makes the transaction example work correctly). I updated the auto_retry example to work with the new for loop finalisation approach, and added an example (reading the lines from multiple named files) where timely iterator finalisation is needed. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam [EMAIL PROTECTED] wrote: Josiah Carlson wrote: It's not a matter of 'will they be finalized', but instead a matter of 'will they be finalized in a timely manner'. From what I understand; upon garbage collection, any generator-based resource will be finalized via __exit__/next(exception)/... and any class-based resource will have its __del__ method called (as long as it is well-behaved), which can be used to call __exit__... I should have said ...should not finalize at the end of the for loop. With generators, you may not want them to finalize before you are done with them, and the same with class's. So you don't use them with a structure that greedily finalizes, and you keep a reference to the object exterior to the loop. Seems to be a non-issue. That is the long-standing nested loops 'issue', which is not going to be solved here, nor should it be. We may not find a solution today, but where should it be addressed if not here? I don't really see the general issue of breaking out of loops as a problem, but was just addressing where it overlaps blocks and weather or not blocks should loop. The argument over whether blocks should loop, I believe has been had; they should. The various use cases involve multi-part transactions and such. The closest thing to a generic solution I can come up with would be to allow for the labeling of for/while loops, and the allowing of break/continue label, which continues to that loop (breaking all other loops currently nested within), or breaks that loop (as well as all other loops currently nested within). Perhaps something like... while ... label 'foo': for ... in ... label 'goo': block ... label 'hoo': if ...: #equivalent to continue 'hoo' continue elif ...: continue 'goo' elif ...: continue 'foo' else: break 'foo' Does this solve the nested loop problem? Yes. Do I like it? Not really; three keywords in a single for/block statement is pretty awful. On the upside, 'label' doesn't need to be a full-on keyword (it can be a partial keyword like 'as' still seems to be). How about this for breaking out of all loops at once. class BreakLoop(Exception): break out of nested loops try: for x in range(100): for y in range(100): for z in range(100): if x == 25 and y==72 and z==3: raise BreakLoop except BreakLoop: pass print 'x,y,z =', x,y,z Sometimes I would like a try until exception: for cases like this where you would use except exception:pass. That is a mechanism, but I like it even less than the one I offered. Every time that one wants ot offer themselves the ability to break out of a different loop (no continue here), one must create another try/except clause, further indenting, and causing nontrivial try/except overhead inside nested loops. A real solution to the problem should (in my opinion) allow the breaking of or continuing to an arbitrary for/while/block. Humorously enough, Richie Hindle's goto/comefrom statements for Python (not to be used in production code) would allow 90% of the necessary behavior (though the lack of timely finalization would probably annoy some people, but then again, there is only so much one can expect from a module written as a working April Fools joke over a year ago). - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Eric Nieuwland [EMAIL PROTECTED] wrote: I suggested to create AN ITERATOR FOR THE LIST and destroy that at the end. The list itself remains untouched. My mistake, I did not understand your use of pronouns. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: Iterating over a sequence. If it's single-pass (and always single pass), you should use a user defined statement instead. That's the technique suggested for the single-pass user defined statements. However, a 'for loop with finalisation' is *still fundamentally an iterative loop*, and the syntax should reflect that. in responce to do The same keyword cannot be used for the looping vs non-looping construct, because of the effect on the semantics of break and continue statements. I disagree with this, I think 'do' would work very well for both single pass, and multiple pass, blocks. In this example 'do' evaluates as True until the generator ends without returning a value: def open_file(name,mode): f = open(name,mode) try: yield f finally: f.close() Do f from open_file(name,mode): for line in f: print line.rstrip() On the first try, it gets f, so the do expression evaluates as True and the BLOCK is run. On the second try, instead of getting a value, the finally suite is executed and the generator ends, causing the do expression to evaluate as False. If a continue is used, it just skips the end of the 'do' body, and then weather or not to loop is determined by weather or not the 'do expression evaluates as True or not. A break skips the rest of the 'do' body and execute the generators finally. This works the same in both single pass and multi pass situations. The difference is by using a truth test instead of iterating, it better represents what is happening and opens up a few options. There's also the possibility to use conditional looping based on the value returned from the generator. do VAR from EXPR if VAR==CONST: BLOCK This is a bit verbose, but it reads well. :-) But that is really just a short cut for: do VAR from EXPR: if VAR != CONST: break BLOCK The Syntax might be: do ([VAR from] EXPR1) | (VAR from EXPR1 if EXPR2): BODY I don't have an opinion on user defined statements yet. But I think they would be somewhat slower than a built in block that does the same thing. What do you mean by 'built in block'? The user defined statements of the PEP redraft are simply a non-looping version of PEP 340's anonymous block statements. Ok, my mistake, I thought you were suggesting the more general user defined statements suggested elsewhere. No, the else clause on loops is a little known part of present day Python - it executes whenever the loop terminates naturally (i.e. not via a break statement). Hmm... ok, and the opposite of what I expected. No wonder its a little known part. My PEP redraft, on the other hand, suggests the introduction of a 'for loop with finalisation' that works fairly similarly to PEP 340's anonymous block statements. Here is my current thinking. It will be better to have 3 separate loops with three identifiable names, and have each work in distinctly different ways. That simplifies, teaching, using, and reading the resulting code. IMHO. 1. For-loops: Fast efficient list iteration. No changes. 2. While-loops: Fast efficient truth test based loop. No changes. 3. Do-loops: An generator based loop with finalization: This could be both single and multiple pass. The difference is determined by weather or not the generator used loops the yield statement or not. I think a good test is the retry example in the PEP. A solution that can represent that clearly and concisely would be a good choice. Maybe this could be made to work: def auto_retry(n, exc): while n0: try: yield True n = 0 except exc: n -= 1 do auto_retry(3, IOError): f = urllib.urlopen(http://python.org/;) print f.read() The ability to propagate the exception back to the generator is what's important here. The while version of this nearly works, but is missing the exception propagation back to the generator, the ability to pass back through the yield, and finalization if the outside while loop is broken before the generator finishes. def auto_retry(n, exc): while n1: try: yield True break except exc: n -= 1 # finalize here yield None import urllib ar = auto_retry(3, IOError) while ar.next(): f = urllib.urlopen(http://python.org/;) print f.read() Although changing 'while' shouldn't be done. I think using 'do' for generator based loops would be good. This isn't that different from PEP340 I think. Maybe it's just comming to the same conclusion from a differnt perspective. shrug :-) Cheers, Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: The argument over whether blocks should loop, I believe has been had; they should. The various use cases involve multi-part transactions and such. Then it is not so much looping but more pushing forward the state of the state of the block's life-cycle? This might by a good moment to consider life-cycle support a la PROCOL. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: [...] The whole PEP draft can be found here: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html [...] Used as follows:: for del auto_retry(3, IOError): f = urllib.urlopen(http://python.org/;) print f.read() I don't know. Using 'del' in that place seems ackward to me. Why not use the following rule: for [VAR in] EXPR: SUITE If EXPR is an iterator, no finalisation is done. If EXPR is not an iterator, it is created at the start and destroyed at the end of the loop. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: You should know why that can't work. If I pass a list, is a list an iterator? No, but it should neither be created nor destroyed before or after. The discussion has been had in regards to why re-using 'for' is a non-starter; re-read the 200+ messages in the thread. - Josiah I agree, re-using or extending 'for' doesn't seem like a good idea to me. I wonder how much effect adding, 'for-next' and the 'StopIteration' exception check as proposed in PEP340, will have on 'for''s performance. And why this isn't just as good: try: for value in iterator: BLOCK1 except StopIteration: BLOCK2 Is one extra line that bad? I think a completely separate looping or non-looping construct would be better for the finalization issue, and maybe can work with class's with __exit__ as well as generators. Having it loop has the advantage of making it break out in a better behaved way. So may be Nicks PEP, would work better with a different keyword? Hint: 'do' Cheers, Ron_Adam ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam [EMAIL PROTECTED] wrote: Josiah Carlson wrote: You should know why that can't work. If I pass a list, is a list an iterator? No, but it should neither be created nor destroyed before or after. The discussion has been had in regards to why re-using 'for' is a non-starter; re-read the 200+ messages in the thread. - Josiah I agree, re-using or extending 'for' doesn't seem like a good idea to me. Now that I've actually stopped to read Nick's PEP, my concern is that 'del', while being a keyword, would not be easy to spot embedded in the rest of the line, and a large number of these 'statements' will only be executed once, so the 'for' may confuse people. I wonder how much effect adding, 'for-next' and the 'StopIteration' exception check as proposed in PEP340, will have on 'for''s performance. For is already tuned to be as fast as possible, which makes sense; it is used 4,523 times in Python 2.4.0's standard library, and easily hundreds of thousands of times in user code. Changing the standard for loop is not to be done lightly. And why this isn't just as good: try: for value in iterator: BLOCK1 except StopIteration: BLOCK2 Is one extra line that bad? I don't know what line you are referring to. I think a completely separate looping or non-looping construct would be better for the finalization issue, and maybe can work with class's with __exit__ as well as generators. From what I understand, the entire conversation has always stated that class-based finalized objects and generator-based finalized objects will both work, and that any proposal that works for one, but not the other, is not sufficient. Having it loop has the advantage of making it break out in a better behaved way. What you have just typed is nonsense. Re-type it and be explicit. Hint: 'do' 'do' has been previously mentioned in the thread. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
On Sun, 08 May 2005 14:16:40 +1000, Nick Coghlan [EMAIL PROTECTED] wrote: Ron Adam wrote: I agree, re-using or extending 'for' doesn't seem like a good idea to me. I agree that re-using a straight 'for' loop is out, due to performance and compatibility issues with applying finalisation semantics to all such iterative loops (there's a reason the PEP redraft doesn't suggest this). However, it makes sense to me that a for loop with finalisation should actually *be* a 'for' loop - just with some extra syntax to indicate that the iterator is finalised at the end of the loop. An option other than the one in my PEP draft would be to put 'del' at the end of the line instead of before EXPR: for [VAR in] EXPR [del]: BLOCK1 else: BLOCK2 However, as you say, 'del' isn't great for the purpose, but I was trying to avoid introduding yet another keyword. An obvious alternative is to use 'finally' instead: for [finally] [VAR in] EXPR: BLOCK1 else: BLOCK2 It still doesn't read all that well, but at least the word more accurately reflects the semantics involved. If such a construct is to be introduced, the ideal spelling would seem to be: for [VAR in] EXPR: BLOCK1 finally: BLOCK2 Jp ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
