Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Greg Ewing wrote: > Nick Coghlan wrote: >> Hmm, with that approach, a code inspection tool like pychecker could >> be used to pick up the slack, and flag generators which have a yield >> inside a try/finally or a user defined statement without applying >> the "needs finalisation" decorator > > What about giving them an __exit__ method if and only > if they have a yield inside a try/finally? Old generators > won't be doing that, because it's currently illegal. It's possible to create a generator that does not contain a finally, but still needs cleanup. def gen(): try: yield except: print 'cleanup' raise Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: > Hmm, with that approach, a code inspection tool like pychecker could be used > to > pick up the slack, and flag generators which have a yield inside a > try/finally > or a user defined statement without applying the "needs finalisation" > decorator What about giving them an __exit__ method if and only if they have a yield inside a try/finally? Old generators won't be doing that, because it's currently illegal. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Greg Ewing wrote: > Unless I'm seriously mistaken, all the Python-equivalent > loop code that's been presented is only for expositional > purposes -- in real life, the logic would be embedded in > the ceval code that implements the for-loop control > bytecodes, so there would be little or no difference in > the bytecode from what is generated today. Hmm, that would obviously be more sensible. OK, I'll drop that from my list of concerns :) > It would be better to do it the other way around, and > have a different form of looping statement for when > you *don't* want finalization. The programmer knows he's > doing a partial iteration when he writes the code, > and is therefore in a position to choose the right > statement. > > For backwards compatibility, the existing for-loop > would work for partial iteration of old iterators, > but this usage would be deprecated. I actually agree, but the pain comes with generators. Doing it this way means that generators can't have an __exit__() method by default - it will need to be enabled with a decorator of some description (a future statement won't work, since it needs to be selectable on a generator by generator basis). It has to be done this way, so that old generators (without the decorator) are not inadvertently finalised by unmodified for loops (otherwise old code that isn't expecting finalisation could break severely). Hmm, with that approach, a code inspection tool like pychecker could be used to pick up the slack, and flag generators which have a yield inside a try/finally or a user defined statement without applying the "needs finalisation" decorator (assuming the compiler can't detect this for itself). OK, given the above, finalising by default definitely seems like the right thing to do - there's then the question of how to spell "don't finalise this iterator". It turns out no keyword is needed for that. Instead, an iterator that iterates over a supplied iterable suffices: class partial_iter(object): def __init__(self, iterable): self.itr = iter(iterable) def __iter__(self): yield self def next(self): return self.itr.next() Then, partial iteration over something that defines __exit__ is possible via: for item in partial_iter(itr): break print list(itr) # itr not finalised, even if it defines __exit__() That reads reasonably well, and it should be possible to reduce the overhead on for loops to a single check of a method slot in the various opcodes that can exit a for loop. So, this idea (or something like it) will go into my next draft. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
[Jim Jewett] > > In case it isn't clear, I think named loops would be a mistake. I > > wanted them when I first started, but ... at the moment, I can't > > think of any usage that wasn't an ugly speed hack, which is at > > least more explicit with the "raise Found" idiom. [Greg Ewing] > I'm inclined to agree. Anything more elaborate than > breaking from a single place in the immediately > enclosing loop tends to be getting into the realm > of spaghetti, in my experience. Giving people named > loops would be tantamount almost to giving them > a goto. Yes please. Stop all discussion of breaking out of multiple loops. It ain't gonna happen before my retirement. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Jim Jewett wrote: > If you're talking specific object instances, then the user is the > only one (outside of the Garbage Collection system) who has a > chance of knowing whether the rest of the iterator will be needed > later. Indeed. He *does* know whether he will want the iterator again later. He *doesn't* know whether it will require finalization when he is eventually done with it, and failing to do so if needed will cause obscure bugs. Also, not needing it again is the overwhelmingly commoner use case. My conclusion is that finalization should be the default, with a way of explicitly overriding it when necessary. > If there are no remaining references, then garbage collection is the > answer, and maybe we just need to make it more aggressive. If > there are remaining references, then maybe the user is wrong about > being done. Or maybe he's not wrong, and due to the way things are coded, the reference happens to hang around a little longer than strictly needed. If garbage collection were sufficient, we'd be relying on it to close our files in the first place, and this whole thread would never have gotten started. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Jim Jewett wrote: > In case it isn't clear, I think named loops would be a mistake. I > wanted them when I first started, but ... at the moment, I can't > think of any usage that wasn't an ugly speed hack, which is at > least more explicit with the "raise Found" idiom. I'm inclined to agree. Anything more elaborate than breaking from a single place in the immediately enclosing loop tends to be getting into the realm of spaghetti, in my experience. Giving people named loops would be tantamount almost to giving them a goto. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: > This bloats the generated byte code horribly, though - it is necessary to > produce two complete copies of the for loop code, since we don't know at > compile > time which version (finalising or non-finalising) will be needed. Unless I'm seriously mistaken, all the Python-equivalent loop code that's been presented is only for expositional purposes -- in real life, the logic would be embedded in the ceval code that implements the for-loop control bytecodes, so there would be little or no difference in the bytecode from what is generated today. > It also takes away from the programmer the ability to choose to do partial > iteration on generators that require finalisation. > > Accordingly, I switched to a version which puts control pack in the hands of > the > programmer. I still think it puts far too much burden on the user, though. The vast majority of the time, for-loops are intended to consume their iterators, and the user may not even know what flavour of iterator is being used, much less want to have to think about it. This means that nearly *every* for-loop would need to have a finally tacked on the end of it as a matter of course. It would be better to do it the other way around, and have a different form of looping statement for when you *don't* want finalization. The programmer knows he's doing a partial iteration when he writes the code, and is therefore in a position to choose the right statement. For backwards compatibility, the existing for-loop would work for partial iteration of old iterators, but this usage would be deprecated. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: > 2. Standard for/while loops should not be finalized in a timely fashion, > because testing for the proper methods would necessarily slow down large > amounts of current Python, so should be left to the garbage collector. I'm not convinced about that. If implemented properly, it could be as simple as testing whether a slot of a type object is populated during processing of the bytecode which causes exit from the loop. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam wrote: > That's sort of why I'm against changing 'for', and for adding > the new loop/block. I see it as a loop with a higher level of > abstraction. A new tool to be used in new ways, but I want to keep my > old dependable tools too. But if there's too much overlap in functionality between the old and the new tool, you're in danger of losing TOOWTDI. Random thought for the day: Programming tools are different from physical tools. I own quite a few different screwdrivers, several of which would be more or less equally good for any particular screw, and this isn't a problem. But I don't have a big crowd of people looking over my shoulder while I work, all trying to figure out why I chose one particular screwdriver over another, and decide which would be the best screwdriver to use on their screws. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam <[EMAIL PROTECTED]> wrote: > Josiah Carlson wrote: > > > I wasn't expressing my opinion, I was attempting to express as to where > > the discussion went and concluded. I honestly can't remember having an > > opinion on the subject, but I seem to have convinced Nick earlier that > > they shouldn't loop, and he (re-)convinced me that indeed, they > > shouldn't loop. > > So you can be re-re-convinced if more use cases are found? ;-) Historically, my opinion has meant close to 0 in regards to language features, so even if I were to change my mind again, I doubt it would have any effect on the outcome of this feature. > >>It needs some polish I think. ;-) > > > > > > Goodness, the horror! When implementation details start bleeding their > > way into actual language constructs (using a continue/break stack in > > order to control the flow of nested loops), that's a good clue that an > > idea has gone a bit too far. > > > > I would honestly prefer gotos, and I would prefer having no change to > > existing syntax to gaining gotos. > > Well I happen to like stacks for somethings. They can be very useful and > efficient. But I agree that they have been horrible abused in all sorts > of ways. But that doesn't make them bad. In this case I think they > work well, but the implementation ccould be improved. You missed my point. As an implementation detail, stacks are fine. As discussion about how one uses it, discussion of stacks is wholly out of the question. Think of it in terms of function calls. Do we talk about 'call stacks' when we make function calls? Of course not, and call stacks are a major part about how Python is implemented. This tells us that we certainly shouldn't talk about 'loop stacks' when we are talking about nested loops. > But this seems awkward and it's even *uglier*! > > try: > # some code in a loop. > except BreakException, Breakme: > pass > > then later > > raise Breakme# breaks a loop now instead of then. That's why we don't see many (if any) uses right now, even if it does solve the nested loop control flow 'problem'. > But these examples sort of point out an underlying concept in these > discussions. That it is useful to be able to postpone or delay code to > be executed at a later time. Thus the idioms, before-after, enter-exit, > and init-finalize. And here again, with try-finally, and breaks/continues. Apple, Apple, Apple, Apple, hotdog. One of those don't quite fit, and that's break/continue. Break/continue statements are about control flow, and while the other mechanisms can be used for control flow, that is not their primary purpose. The first three are about resource acquisition/release and try/finally is about making sure that a block of code is executed "no matter what" (the first three using try/finally as a mechanism to guarantee resource release, if acquisition has taken place). Let us look at the flow of the conversation... "There should be a resource acquisition/release statement with a code suite attached." ... lead to ... "The behavior of a break within such a code suite is ambiguous." ... lead to ... "Flow within nested loops themselves can be tricky and/or ambiguous." It's not that there is an underlying concept, it's just that the third bit of the conversation hadn't been brought up, but 'solving' that bit would unambiguate the previous 'break within code block' ambiguity. It's a sledge hammer solution to a jewel hammer problem. > > It's kind of funny. Every month I spend in python-dev, I feel less > > inclined to want to change the Python language (except for the relative > > import I need to finish implementing). Not because it is a pain in the > > tookus (though it is), but because many times it is my immediate sense > > of aesthetics that causes me to desire change, and my future of code > > maintenance makes me think forward to understanding Python 2.3 in the > > context of Python 2.9 . > > Changing it just for the sake of newness or "it's cool" or whatever > isn't good of course. In general, I was thinking "that would make my life as a programmer working with Python easier", but within a few days I realize that Python's power is in its simplicity. With every new language ability/feature, learning and using the language (necessarily) becomes more difficult. > But extending a language vs changing the way it > works, is worth while as long as it's done in a careful and consistent > manner. That's sort of why I'm against changing 'for', and for adding > the new loop/block. I see it as a loop with a higher level of > abstraction. A new tool to be used in new ways, but I want to keep my > old dependable tools too. I'm not sure that the discussion has been leading toward a change that is 'careful and consistent', but more of people offering their pet desired syntaxes (I've been having dejavu to the decorator discussion of last spring/summer). Of course the only
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Raymond Hettinger wrote: > [Nick Coghlan] > >>The number of good use cases for a looping block statement currently >>stands at >>exactly 1 (auto_retry). Every other use case suggested (locking, > > opening, > >>suppressing, etc) involves factoring out try statement boiler plate > > that > >>is far >>easier to comprehend with a single pass user defined statement. > > > I would like to offer up one additional use case, eliminating redundant > code in a do-while construct: > > > def do_while(cond_func): > yield > while cond_func(): > yield > > block do_while(lambda: a>b): > Nice example, but it doesn't need to intercept exceptions the way auto_retry does. Accordingly, a 'for' loop which makes [VAR in] optional would do the job just fine: for do_while(lambda: a>b): Even today, the following works: def do_while(cond_func): yield None while cond_func(): yield None for _ in do_while(lambda: a>b): Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: >> Ron Adam <[EMAIL PROTECTED]> wrote: >>Yes, so just "do [VAR from] EXPR1:" > > Regardless of the 'finalization' syntax, I'm talking about the fact that > including extra 'if EXPR' or 'while EXPR' is not going to be an option. Yes, I meant for the syntax to be the shorter form, not for the programmer to just leave off the end. >>But isn't this what PEP340 *already* proposes? Or am I missing a subtle >>distinction here. > > It is, in fact, what PEP 340 already proposes. Let us take a step back > for a moment and realize that this entire discussion is going around in > circles. I think so too. > From what I understand, we all agree: > 1. There should be some mechanism X which signals that an indented suite > is a 'block statement'. Such blocks are described and finalized as per > PEP 340 (or whatever derivative gets accepted). +1 > 2. Standard for/while loops should not be finalized in a timely fashion, > because testing for the proper methods would necessarily slow down large > amounts of current Python, so should be left to the garbage collector. +1 Also add to this, it is not always desirable to finalize an object after use in a for loop. > 3. A note as to the additional overhead of finalized blocks should be > mentioned in the source code of the finalization implementation, and a > note on their performace characteristics may or may not need to be > listed in the language reference. +1 > What there is still discussion over: > 4. What the syntax should be. > 5. Whether or not it loops. > 6. Method names. > > I find the answers to 4,5,6 to be a matter of opinion, and like many, I > have my own. However, I do not feel strongly enough about 4,5,6 to > argue about my opinion (I've been attempting to re-state where the > conversation went for the last 2 weeks, as I have managed to read each > and every email about the subject at hand *ick*). > > - Josiah I think you clarified this well. Item 4: A list of possible syntax's with a vote at some point should do. Item 5: (A.) More use case's for looping blocks need to be found. I think there may be some or many that are not obvious at the moment. (B.) It may not cost much in performance to include the looping behavior. Maybe this should be put off till there is a working version of each, then comparisons of performance can be made in different situations? _Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam <[EMAIL PROTECTED]> wrote: > > Josiah Carlson wrote: > > > Ron Adam <[EMAIL PROTECTED]> wrote: > > > >> There's also the possibility to use conditional looping based on the > value returned from the generator. > >> > >> do VAR from EXPR if VAR==CONST: > >>BLOCK > >> > >> This is a bit verbose, but it reads well. :-) > > > > > > > > Reading well or not, this is not really an option for the same reasons > > why... > > > > for VAR in EXPR1 if EXPR2: > > or > > for VAR in EXPR1 while EXPR2: > > > > are not options. Keep it simple. > > > > Yes, so just "do [VAR from] EXPR1:" Regardless of the 'finalization' syntax, I'm talking about the fact that including extra 'if EXPR' or 'while EXPR' is not going to be an option. > >>3. Do-loops: An generator based loop with finalization: This > could be both single and multiple pass. The difference is determined by > weather or not the generator used loops the yield statement or not. > > > > > > Offering only generator-based finalization loops is, as I understand it, > > not an option. > > It could also include classes with __exit__ methods which are really > just more complex generators when used this way. > > But isn't this what PEP340 *already* proposes? Or am I missing a subtle > distinction here. It is, in fact, what PEP 340 already proposes. Let us take a step back for a moment and realize that this entire discussion is going around in circles. >From what I understand, we all agree: 1. There should be some mechanism X which signals that an indented suite is a 'block statement'. Such blocks are described and finalized as per PEP 340 (or whatever derivative gets accepted). 2. Standard for/while loops should not be finalized in a timely fashion, because testing for the proper methods would necessarily slow down large amounts of current Python, so should be left to the garbage collector. 3. A note as to the additional overhead of finalized blocks should be mentioned in the source code of the finalization implementation, and a note on their performace characteristics may or may not need to be listed in the language reference. What there is still discussion over: 4. What the syntax should be. 5. Whether or not it loops. 6. Method names. I find the answers to 4,5,6 to be a matter of opinion, and like many, I have my own. However, I do not feel strongly enough about 4,5,6 to argue about my opinion (I've been attempting to re-state where the conversation went for the last 2 weeks, as I have managed to read each and every email about the subject at hand *ick*). - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Jim Jewett <[EMAIL PROTECTED]> wrote: > > Nick Coghlan wrote: > > Josiah Carlson wrote: > > >> This has the benefit that an arbitrary block of code can be named, > >> and a named TerminateBlock used to exit it. > > >> ... I suspect any such implementation is going to need to use > >> exceptions for the guts of the flow control, even if that use isn't > >> visible to the programmer. > > > Not necessarily. If I were implementing such a thing; any time > > arbitrary break/continues (to a loop that isn't the deepest) were used > > in nested loops, I would increment a counter any time a loop was entered, > > and decrement the counter any time a loop was exited. ... > > When named blocks are used in Lisp, they often cross function > boundaries. Given that, the number of intervening loops could > change depending on external variables. Since you would have > to pop frames anyhow, Exceptions are the right way to do it. I wasn't talking about cross-function blocks/named blocks. I was strictly talking about nested loops as they currently exist in Python. > If you limited the named-block gotos to within a single function/method, > then the loop counter would work (and you could limit obfuscation). > Unfortunately, you would lose most of the power of named blocks, > while still paying the full ugliness price. That's fine, I don't want named loops or blocks anyhow. I was merely offering an implementation that did not require exceptions, and was necessarily fast (proving both that it could be fast and not require exceptions). > You would also encourage > people to inline things that ought to be separate functions. I wouldn't go that far. If one were to introduce such functionality, it would be to ease control flow within nested for/while/blocks. Whether or not that lead to people inlining code, who are we to say? It would, however, complicate the 'inline function' decorator that I seem to have lost my link to. > In case it isn't clear, I think named loops would be a mistake. I > wanted them when I first started, but ... at the moment, I can't > think of any usage that wasn't an ugly speed hack, which is at > least more explicit with the "raise Found" idiom. Don't get me wrong, I think they would be a mistake as well, but they would solve the 'does a break statement in a block break its enclosing loop' question, as well as general nested loop flow control issues. Now that we both agree that they shouldn't be done, maybe one of us should write a PEP for Guido to rule on so that we never have to hear about loop naming (heh). - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: > I wasn't expressing my opinion, I was attempting to express as to where > the discussion went and concluded. I honestly can't remember having an > opinion on the subject, but I seem to have convinced Nick earlier that > they shouldn't loop, and he (re-)convinced me that indeed, they > shouldn't loop. So you can be re-re-convinced if more use cases are found? ;-) >>It needs some polish I think. ;-) > > > Goodness, the horror! When implementation details start bleeding their > way into actual language constructs (using a continue/break stack in > order to control the flow of nested loops), that's a good clue that an > idea has gone a bit too far. > > I would honestly prefer gotos, and I would prefer having no change to > existing syntax to gaining gotos. Well I happen to like stacks for somethings. They can be very useful and efficient. But I agree that they have been horrible abused in all sorts of ways. But that doesn't make them bad. In this case I think they work well, but the implementation ccould be improved. I see a correlation between breaks, continues, and exceptions. So it makes since to me that they could use similar mechanisms to catch and reraise them as needed. But this seems awkward and it's even *uglier*! try: # some code in a loop. except BreakException, Breakme: pass then later raise Breakme# breaks a loop now instead of then. But these examples sort of point out an underlying concept in these discussions. That it is useful to be able to postpone or delay code to be executed at a later time. Thus the idioms, before-after, enter-exit, and init-finalize. And here again, with try-finally, and breaks/continues. _Ron > It's kind of funny. Every month I spend in python-dev, I feel less > inclined to want to change the Python language (except for the relative > import I need to finish implementing). Not because it is a pain in the > tookus (though it is), but because many times it is my immediate sense > of aesthetics that causes me to desire change, and my future of code > maintenance makes me think forward to understanding Python 2.3 in the > context of Python 2.9 . Changing it just for the sake of newness or "it's cool" or whatever isn't good of course. But extending a language vs changing the way it works, is worth while as long as it's done in a careful and consistent manner. That's sort of why I'm against changing 'for', and for adding the new loop/block. I see it as a loop with a higher level of abstraction. A new tool to be used in new ways, but I want to keep my old dependable tools too. _Ron The simplest computer language in the world is called MAIN, has only one function called main(), which evaluates any arbitrary set of arguments you give it. Programming is now an obsolete profession! However we are now in desperate need of experts who can understand exceedingly long arbitrary sets of arguments and what they will do when they are evaluated by MAIN's main() function. ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Greg Ewing wrote: > Ron Adam wrote: > >>There seems to be some confusion as to weather or >>not 'for's will do finalizing. So I was trying to stress I think >>regular 'for' loops should not finalize. They should probably give an >>error if an object with an try-finally in them or an __exit__ method. > > > But if the for-loop can tell whether the iterator > needs finalizing or not, why not have it finalize > the ones that need it and not finalize the ones > that don't? That would be backwards compatible, > since old for-loops working on old iterators would > work as before. > > Greg Ok, so if they check for it, they might as well handle it. Or maybe they shouldn't even check for performance reasons? Then no error, and it's up to the programmer to decide which looping construct to use. _Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Greg Ewing wrote: > Ron Adam wrote: >> There seems to be some confusion as to weather or >> not 'for's will do finalizing. So I was trying to stress I think >> regular 'for' loops should not finalize. They should probably give an >> error if an object with an try-finally in them or an __exit__ method. > > But if the for-loop can tell whether the iterator > needs finalizing or not, why not have it finalize > the ones that need it and not finalize the ones > that don't? That would be backwards compatible, > since old for-loops working on old iterators would > work as before. That's why I suggested to have the behaviour depend on what is passed in as EXPR. for VAR in EXPR: BLOCK could be translated to: __cleanup = False __itr = EXPR if not isinstance(__itr,iterator): __itr = iter(__itr) __cleanup = True while True: try: VAR = __itr.next() except StopIteration: break BLOCK if __cleanup: __itr.__exit__() Which would require isinstance(__itr,iterator) or equivalent to act as a robust test on iterators. I'll leave 'for' with an 'else' clause as an exercise to the reader. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: > Ron Adam <[EMAIL PROTECTED]> wrote: > >> There's also the possibility to use conditional looping based on the value returned from the generator. >> >> do VAR from EXPR if VAR==CONST: >>BLOCK >> >> This is a bit verbose, but it reads well. :-) > > > > Reading well or not, this is not really an option for the same reasons > why... > > for VAR in EXPR1 if EXPR2: > or > for VAR in EXPR1 while EXPR2: > > are not options. Keep it simple. > Yes, so just "do [VAR from] EXPR1:" >>3. Do-loops: An generator based loop with finalization: This could be both single and multiple pass. The difference is determined by weather or not the generator used loops the yield statement or not. > > > Offering only generator-based finalization loops is, as I understand it, > not an option. It could also include class's with __exit__ methods which are really just more complex generators when used this way. But isn't this what PEP340 *already* proposes? Or am I missing a subtle distinction here. -Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: > "Loop on this iterator and finalise when done" would be written: >for item in itr: >process(item) >finally: >pass Greg Ewing wrote: > This is approaching things from the wrong end. The user of > an iterator shouldn't need to know or care whether it > requires finalization -- it should Just Work, whatever > context it is used in. If you're saying "lists don't need it, but openfiles do", then I agree; it shouldn't matter what type of iterator you have. If you're talking specific object instances, then the user is the only one (outside of the Garbage Collection system) who has a chance of knowing whether the rest of the iterator will be needed later. When iterating over lines in a file, and breaking out at a sentinel, the compiler can't know whether you're done, or just leaving the "real" lines to another piece of code. Of course, that still raises the "Why are we encouraging bugs?" issue. If there are no remaining references, then garbage collection is the answer, and maybe we just need to make it more aggressive. If there are remaining references, then maybe the user is wrong about being done. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: > Josiah Carlson wrote: >> This has the benefit that an arbitrary block of code can be named, >> and a named TerminateBlock used to exit it. >> ... I suspect any such implementation is going to need to use >> exceptions for the guts of the flow control, even if that use isn't >> visible to the programmer. > Not necessarily. If I were implementing such a thing; any time > arbitrary break/continues (to a loop that isn't the deepest) were used > in nested loops, I would increment a counter any time a loop was entered, > and decrement the counter any time a loop was exited. ... When named blocks are used in Lisp, they often cross function boundaries. Given that, the number of intervening loops could change depending on external variables. Since you would have to pop frames anyhow, Exceptions are the right way to do it. If you limited the named-block gotos to within a single function/method, then the loop counter would work (and you could limit obfuscation). Unfortunately, you would lose most of the power of named blocks, while still paying the full ugliness price. You would also encourage people to inline things that ought to be separate functions. In case it isn't clear, I think named loops would be a mistake. I wanted them when I first started, but ... at the moment, I can't think of any usage that wasn't an ugly speed hack, which is at least more explicit with the "raise Found" idiom. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Greg Ewing wrote: > Ron Adam wrote: > >>There seems to be some confusion as to weather or >>not 'for's will do finalizing. So I was trying to stress I think >>regular 'for' loops should not finalize. They should probably give an >>error if an object with an try-finally in them or an __exit__ method. > > > But if the for-loop can tell whether the iterator > needs finalizing or not, why not have it finalize > the ones that need it and not finalize the ones > that don't? That would be backwards compatible, > since old for-loops working on old iterators would > work as before. When I first started redrafting the PEP, I had essentially this idea in there - look for an __exit__() method, if it's there use the new 'finalising' semantics, if it isn't, use the old semantics. This bloats the generated byte code horribly, though - it is necessary to produce two complete copies of the for loop code, since we don't know at compile time which version (finalising or non-finalising) will be needed. It also takes away from the programmer the ability to choose to do partial iteration on generators that require finalisation. And it does so in a non-obvious way: "it works with this iterator, why doesn't it work with that one?" Accordingly, I switched to a version which puts control pack in the hands of the programmer. If you know the iterator doesn't need finalising, or if eventual finalisation on garbage collection is sufficient, then you can omit the finally clause, and get the optimised form of the for loop. Alternatively, if you want prompt finalisation (e.g. with an iterator like all_lines() from the PEP redraft), then you can include the finally clause and get the behaviour you want. Failure to finalise promptly on iterators that need it is still a bug - but then, so is failing to close a file handle or database connection. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: > "Loop on this iterator and finalise when done" would be written: > >for item in itr: >process(item) >finally: >pass This is approaching things from the wrong end. The user of an iterator shouldn't need to know or care whether it requires finalization -- it should Just Work, whatever context it is used in. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam <[EMAIL PROTECTED]> wrote: > > The argument over whether blocks should loop, I believe has been had; > > they should. The various use cases involve multi-part transactions and > > such. > > I think so now too, I had thought as Nick does earlier this week that > the non-looping version was cleaner, but changed my mind when I realized > that looping blocks could be made to work for those in a simple and > understandable way. I wasn't expressing my opinion, I was attempting to express as to where the discussion went and concluded. I honestly can't remember having an opinion on the subject, but I seem to have convinced Nick earlier that they shouldn't loop, and he (re-)convinced me that indeed, they shouldn't loop. > I think maybe another alternative is a break buffer or cue. Where you > push a 'break' onto the buffer and then execute a 'break' to break the > current loop, The 'break' in the buffer then breaks the next loop out as > soon as the current loop exits, etc. [snip] > It needs some polish I think. ;-) Goodness, the horror! When implementation details start bleeding their way into actual language constructs (using a continue/break stack in order to control the flow of nested loops), that's a good clue that an idea has gone a bit too far. I would honestly prefer gotos, and I would prefer having no change to existing syntax to gaining gotos. It's kind of funny. Every month I spend in python-dev, I feel less inclined to want to change the Python language (except for the relative import I need to finish implementing). Not because it is a pain in the tookus (though it is), but because many times it is my immediate sense of aesthetics that causes me to desire change, and my future of code maintenance makes me think forward to understanding Python 2.3 in the context of Python 2.9 . - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam wrote: > There seems to be some confusion as to weather or > not 'for's will do finalizing. So I was trying to stress I think > regular 'for' loops should not finalize. They should probably give an > error if an object with an try-finally in them or an __exit__ method. But if the for-loop can tell whether the iterator needs finalizing or not, why not have it finalize the ones that need it and not finalize the ones that don't? That would be backwards compatible, since old for-loops working on old iterators would work as before. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan <[EMAIL PROTECTED]> wrote: > Josiah Carlson wrote: > > The argument over whether blocks should loop, I believe has been had; > > they should. The various use cases involve multi-part transactions and > > such. [snip looping block discussion] > For the one good use case for a user defined loop (auto_retry), I initially > suggested in my redraft that there be a way of denoting that a given for loop > gives the iterator the opportunity to intercept exceptions raised in the body > of > the loop (like the PEP 340 block statement). You convinced me that was a bad > idea, and I switched to a simple iterator finalisation clause in version 1.2. Well then, I guess you have re-convinced me that the block statement probably shouldn't loop. > Even with that simplified approach though, *using* auto_retry is still very > easy: > >for attempt in auto_retry(3, IOError): >stmt attempt: >do_something() > > It's a little trickier to write auto_retry itself, since you can't easily use > a > generator anymore, but it still isn't that hard, and the separation of > concerns > (between iteration, and the customised control flow in response to > exceptions) > makes it very easy to grasp how it works. Great. Now all we need is a module with a handful of finalization generators, with all of the obvious ones already implemented. > >>>The closest thing to a generic solution I can come > >>>up with would be to allow for the labeling of for/while loops, and the > >>>allowing of "break/continue ", which continues to that loop > >>>(breaking all other loops currently nested within), or breaks that loop > >>>(as well as all other loops currently nested within). > > Or, we simply have user defined statements which are not themselves loops, > and > use them to create named blocks: [snipped code to protect the innocent] > This has the benefit that an arbitrary block of code can be named, and a > named > TerminateBlock used to exit it. Scary. > > That is a mechanism, but I like it even less than the one I offered. > > Every time that one wants ot offer themselves the ability to break out > > of a different loop (no continue here), one must create another > > try/except clause, further indenting, and causing nontrivial try/except > > overhead inside nested loops. > > Ah well, that criticism applies to my suggestion, too. However, I suspect any > such implementation is going to need to use exceptions for the guts of the > flow > control, even if that use isn't visible to the programmer. Not necessarily. If I were implementing such a thing; any time arbitrary break/continues (to a loop that isn't the deepest) were used in nested loops, I would increment a counter any time a loop was entered, and decrement the counter any time a loop was exited. When performing a break/continue, I would merely set another variable for which loop is the final break/continue, then the interpreter could break loops while the desired level/current level differed, then perform a final break/continue depending on what was executed. No exceptions necessary, and the increment/decrement should necessarily be cheap (an increment/decrement of a char, being that Python limits itself to 20 nested fors, and probably should limit itself to X nested loops, where X < 256). - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam <[EMAIL PROTECTED]> wrote: > There's also the possibility to use conditional looping based on the > value returned from the generator. > > do VAR from EXPR if VAR==CONST: > BLOCK > > This is a bit verbose, but it reads well. :-) Reading well or not, this is not really an option for the same reasons why... for VAR in EXPR1 if EXPR2: or for VAR in EXPR1 while EXPR2: are not options. Keep it simple. > 3. Do-loops: An generator based loop with finalization: This could > be both single and multiple pass. The difference is determined by > weather or not the generator used loops the yield statement or not. Offering only generator-based finalization loops is, as I understand it, not an option. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote:
> Ron Adam <[EMAIL PROTECTED]> wrote:
>>I should have said "...should not finalize at the end of the for loop".
>> With generators, you may not want them to finalize before you are done
>>with them, and the same with class's.
>
>
> So you don't use them with a structure that greedily finalizes, and you
> keep a reference to the object exterior to the loop. Seems to be a
> non-issue.
Yes, it should be a non issue.
> The argument over whether blocks should loop, I believe has been had;
> they should. The various use cases involve multi-part transactions and
> such.
I think so now too, I had thought as Nick does earlier this week that
the non-looping version was cleaner, but changed my mind when I realized
that looping blocks could be made to work for those in a simple and
understandable way.
>>try:
>> for x in range(100):
>> for y in range(100):
>> for z in range(100):
>> if x == 25 and y==72 and z==3:
>> raise BreakLoop
>>
>>except BreakLoop: pass
>>print 'x,y,z =', x,y,z
> That is a mechanism, but I like it even less than the one I offered.
> Every time that one wants ot offer themselves the ability to break out
> of a different loop (no continue here), one must create another
> try/except clause, further indenting, and causing nontrivial try/except
> overhead inside nested loops.
>
> A real solution to the problem should (in my opinion) allow the breaking
> of or continuing to an arbitrary for/while/block. Humorously enough,
> Richie Hindle's goto/comefrom statements for Python ("not to be used in
> production code") would allow 90% of the necessary behavior (though the
> lack of timely finalization would probably annoy some people, but then
> again, there is only so much one can expect from a module written as a
> working April Fools joke over a year ago).
>
> - Josiah
I think maybe another alternative is a break buffer or cue. Where you
push a 'break' onto the buffer and then execute a 'break' to break the
current loop, The 'break' in the buffer then breaks the next loop out as
soon as the current loop exits, etc.
for x in range(100):
for y in range(100):
for z in range(100):
if x == 25 and y==72 and z==3:
push_loop(Break,Break) # will break both parent loops
break # break current loop
if push_loop(...) could take a NoBreak, then you can selectively break
outer breaks by how you sequence them.
push_break(None, break) wound not break the y loop, but will break the x
loop above. Can you think of a use case for something like that?
Pushing 'Continues' might also work:
for x in range(100):
for y in range(100):
if x == 25 and y==72:
push_loop(Continue) # will skip rest of parents body
break# break current loop
#code2
#code1
This will break the 'y' loop and skip code2, then continue the 'x' loop
skipping code block 1.
Using a stack for breaks and continues isn't too different than using a
stack for exceptions I think.
Also by making it a function call instead of a command you can have a
function return a Break or Continue object, or None,
for x in range(100):
for y in range(100):
if y == testval:
push_loop(looptest(x,y)):break x loop depending on x,y
break
None's returned would need to be discarded I think for this to work, so
something else would be needed to skip a level.
It needs some polish I think. ;-)
Cheers,
Ron_Adam
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote:
> The argument over whether blocks should loop, I believe has been had;
> they should. The various use cases involve multi-part transactions and
> such.
The number of good use cases for a looping block statement currently stands at
exactly 1 (auto_retry). Every other use case suggested (locking, opening,
suppressing, etc) involves factoring out try statement boiler plate that is far
easier to comprehend with a single pass user defined statement. A single pass
user defined statement allows all such code to be factored safely, even if the
main clause of the try statement uses break or continue statements.
The insanity of an inherently looping block statement is shown by the massive
semantic differences between the following two pieces of code under PEP 340:
block locking(the_lock):
for item in items:
if handle(item):
break
for item in items:
block locking(the_lock):
if handle(item):
break
With a non-looping user defined statement, you get the semantics you would
expect for the latter case (i.e. the for loop is still terminated after an item
is handled, whereas that won't happen under PEP 340)
For the one good use case for a user defined loop (auto_retry), I initially
suggested in my redraft that there be a way of denoting that a given for loop
gives the iterator the opportunity to intercept exceptions raised in the body
of
the loop (like the PEP 340 block statement). You convinced me that was a bad
idea, and I switched to a simple iterator finalisation clause in version 1.2.
Even with that simplified approach though, *using* auto_retry is still very
easy:
for attempt in auto_retry(3, IOError):
stmt attempt:
do_something()
It's a little trickier to write auto_retry itself, since you can't easily use a
generator anymore, but it still isn't that hard, and the separation of concerns
(between iteration, and the customised control flow in response to exceptions)
makes it very easy to grasp how it works.
>>>The closest thing to a generic solution I can come
>>>up with would be to allow for the labeling of for/while loops, and the
>>>allowing of "break/continue ", which continues to that loop
>>>(breaking all other loops currently nested within), or breaks that loop
>>>(as well as all other loops currently nested within).
Or, we simply have user defined statements which are not themselves loops, and
use them to create named blocks:
def block(name):
try:
yield
except TerminateBlock, ex:
if not ex.args or ex.args[0] != name
raise
stmt block('foo'):
while condition():
stmt block('goo'):
for ... in ...:
while other_case():
stmt block('hoo'):
if ...:
# Continue the inner while loop
continue
if ...:
# Exit the inner while loop
raise TerminateBlock, 'hoo'
if ...:
# Exit the for loop
raise TerminateBlock, 'goo'
# Exit the outer while loop
raise TerminateBlock, 'foo'
This has the benefit that an arbitrary block of code can be named, and a named
TerminateBlock used to exit it.
> That is a mechanism, but I like it even less than the one I offered.
> Every time that one wants ot offer themselves the ability to break out
> of a different loop (no continue here), one must create another
> try/except clause, further indenting, and causing nontrivial try/except
> overhead inside nested loops.
Ah well, that criticism applies to my suggestion, too. However, I suspect any
such implementation is going to need to use exceptions for the guts of the flow
control, even if that use isn't visible to the programmer.
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Jp Calderone <[EMAIL PROTECTED]> writes: > If such a construct is to be introduced, the ideal spelling would seem to > be: > > for [VAR in] EXPR: > BLOCK1 > finally: > BLOCK2 Does this mean that adding finally: pass to a for block would make the for loop behave differently? Cheers, mwh -- I really hope there's a catastrophic bug in some future e-mail program where if you try and send an attachment it cancels your ISP account, deletes your harddrive, and pisses in your coffee -- Adam Rixey ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: > The argument over whether blocks should loop, I believe has been had; > they should. The various use cases involve multi-part transactions and > such. Then it is not so much looping but more pushing forward the state of the state of the block's life-cycle? This might by a good moment to consider life-cycle support a la PROCOL. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: > Eric Nieuwland <[EMAIL PROTECTED]> wrote: >> I suggested to create AN ITERATOR FOR THE LIST and destroy that at the >> end. The list itself remains untouched. > > My mistake, I did not understand your use of pronouns. And, rereading my post, I used an ambigous reference. My bad as well. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote:
> Iterating over a sequence. If it's single-pass (and always single pass), you
> should use a user defined statement instead.
> That's the technique suggested for the single-pass user defined statements.
> However, a 'for loop with finalisation' is *still fundamentally an iterative
> loop*, and the syntax should reflect that.
> The same keyword cannot be used for the looping vs non-looping construct,
> because of the effect on the semantics of break and continue statements.
I disagree with this, I think 'do' would work very well for both single
pass, and multiple pass, blocks.
In this example 'do' evaluates as True until the generator ends without
returning a value:
def open_file(name,mode):
f = open(name,mode)
try:
yield f
finally:
f.close()
Do f from open_file(name,mode):
for line in f:
print line.rstrip()
On the first try, it gets f, so the do expression evaluates as True and
the BLOCK is run.
On the second try, instead of getting a value, the finally suite is
executed and the generator ends, causing the do expression to evaluate
as False.
If a continue is used, it just skips the end of the 'do' body, and then
weather or not to loop is determined by weather or not the 'do
expression evaluates as True or not.
A break skips the rest of the 'do' body and execute the generators
finally.
This works the same in both single pass and multi pass situations.
The difference is by using a truth test instead of iterating, it better
represents what is happening and opens up a few options.
There's also the possibility to use conditional looping based on the
value returned from the generator.
do VAR from EXPR if VAR==CONST:
BLOCK
This is a bit verbose, but it reads well. :-)
But that is really just a short cut for:
do VAR from EXPR:
if VAR != CONST:
break
BLOCK
The Syntax might be:
do ([VAR from] EXPR1) | (VAR from EXPR1 if EXPR2): BODY
>>I don't have an opinion on user defined statements yet. But I think
>>they would be somewhat slower than a built in block that does the same
>>thing.
>
> What do you mean by 'built in block'? The user defined statements of the PEP
> redraft are simply a non-looping version of PEP 340's anonymous block
> statements.
Ok, my mistake, I thought you were suggesting the more general user
defined statements suggested elsewhere.
> No, the else clause on loops is a little known part of present day Python -
> it
> executes whenever the loop terminates naturally (i.e. not via a break
> statement).
Hmm... ok, and the opposite of what I expected. No wonder its a little
known part.
> My PEP redraft, on the other hand, suggests the introduction of a 'for loop
> with
> finalisation' that works fairly similarly to PEP 340's anonymous block
> statements.
Here is my current thinking. It will be better to have 3 separate loops
with three identifiable names, and have each work in distinctly
different ways. That simplifies, teaching, using, and reading the
resulting code. IMHO.
1. For-loops: Fast efficient list iteration. No changes.
2. While-loops: Fast efficient truth test based loop. No changes.
3. Do-loops: An generator based loop with finalization: This could
be both single and multiple pass. The difference is determined by
weather or not the generator used loops the yield statement or not.
I think a good test is the retry example in the PEP. A solution that
can represent that clearly and concisely would be a good choice.
Maybe this could be made to work:
def auto_retry(n, exc):
while n>0:
try:
yield True
n = 0
except exc:
n -= 1
do auto_retry(3, IOError):
f = urllib.urlopen("http://python.org/";)
print f.read()
The ability to propagate the exception back to the generator is what's
important here.
The while version of this nearly works, but is missing the exception
propagation back to the generator, the ability to pass back through the
yield, and finalization if the outside while loop is broken before the
generator finishes.
def auto_retry(n, exc):
while n>1:
try:
yield True
break
except exc:
n -= 1
# finalize here
yield None
import urllib
ar = auto_retry(3, IOError)
while ar.next():
f = urllib.urlopen("http://python.org/";)
print f.read()
Although changing 'while' shouldn't be done. I think using 'do' for
generator based loops would be good.
This isn't that different from PEP340 I think. Maybe it's just comming
to the same conclusion from a differnt perspective. :-)
Cheers, Ron
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Eric Nieuwland <[EMAIL PROTECTED]> wrote: > I suggested to create AN ITERATOR FOR THE LIST and destroy that at the > end. The list itself remains untouched. My mistake, I did not understand your use of pronouns. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam <[EMAIL PROTECTED]> wrote:
> Josiah Carlson wrote:
> > It's not a matter of 'will they be finalized', but instead a matter of
> > 'will they be finalized in a timely manner'. From what I understand;
> > upon garbage collection, any generator-based resource will be finalized
> > via __exit__/next(exception)/... and any class-based resource will have
> > its __del__ method called (as long as it is well-behaved), which can be
> > used to call __exit__...
>
> I should have said "...should not finalize at the end of the for loop".
> With generators, you may not want them to finalize before you are done
> with them, and the same with class's.
So you don't use them with a structure that greedily finalizes, and you
keep a reference to the object exterior to the loop. Seems to be a
non-issue.
> > That is the long-standing nested loops 'issue', which is not going to be
> > solved here, nor should it be.
>
> We may not find a solution today, but where should it be addressed if
> not here?
>
> I don't really see the general issue of breaking out of loops as a
> problem, but was just addressing where it overlaps blocks and weather or
> not blocks should loop.
The argument over whether blocks should loop, I believe has been had;
they should. The various use cases involve multi-part transactions and
such.
> > The closest thing to a generic solution I can come
> > up with would be to allow for the labeling of for/while loops, and the
> > allowing of "break/continue ", which continues to that loop
> > (breaking all other loops currently nested within), or breaks that loop
> > (as well as all other loops currently nested within).
> >
> > Perhaps something like...
> >
> > while ... label 'foo':
> > for ... in ... label 'goo':
> > block ... label 'hoo':
> > if ...:
> > #equivalent to continue 'hoo'
> > continue
> > elif ...:
> > continue 'goo'
> > elif ...:
> > continue 'foo'
> > else:
> > break 'foo'
> >
> > Does this solve the nested loop problem? Yes. Do I like it? Not
> > really; three keywords in a single for/block statement is pretty awful.
> > On the upside, 'label' doesn't need to be a full-on keyword (it can be a
> > partial keyword like 'as' still seems to be).
>
> How about this for breaking out of all loops at once.
>
> class BreakLoop(Exception):
> """break out of nested loops"""
>
> try:
> for x in range(100):
> for y in range(100):
> for z in range(100):
> if x == 25 and y==72 and z==3:
> raise BreakLoop
>
> except BreakLoop: pass
> print 'x,y,z =', x,y,z
>
>
> Sometimes I would like a "try until :" for cases like this
> where you would use "except :pass".
That is a mechanism, but I like it even less than the one I offered.
Every time that one wants ot offer themselves the ability to break out
of a different loop (no continue here), one must create another
try/except clause, further indenting, and causing nontrivial try/except
overhead inside nested loops.
A real solution to the problem should (in my opinion) allow the breaking
of or continuing to an arbitrary for/while/block. Humorously enough,
Richie Hindle's goto/comefrom statements for Python ("not to be used in
production code") would allow 90% of the necessary behavior (though the
lack of timely finalization would probably annoy some people, but then
again, there is only so much one can expect from a module written as a
working April Fools joke over a year ago).
- Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: > Ron Adam <[EMAIL PROTECTED]> wrote: > >>Josiah Carlson wrote: >> I think a completely separate looping or non-looping construct would be better for the finalization issue, and maybe can work with class's with __exit__ as well as generators. >>> >>>From what I understand, the entire conversation has always stated that >>>class-based finalized objects and generator-based finalized objects will >>>both work, and that any proposal that works for one, but not the other, >>>is not sufficient. >> >>That's good to hear. There seems to be some confusion as to weather or >>not 'for's will do finalizing. So I was trying to stress I think >>regular 'for' loops should not finalize. They should probably give an >>error if an object with an try-finally in them or an __exit__ method. >>I'm not sure what the current opinion on that is. But I didn't see it >>in any of the PEPs. > > > It's not a matter of 'will they be finalized', but instead a matter of > 'will they be finalized in a timely manner'. From what I understand; > upon garbage collection, any generator-based resource will be finalized > via __exit__/next(exception)/... and any class-based resource will have > its __del__ method called (as long as it is well-behaved), which can be > used to call __exit__... I should have said "...should not finalize at the end of the for loop". With generators, you may not want them to finalize before you are done with them, and the same with class's. Having it loop has the advantage of making it break out in a better behaved way. >>> >>>What you have just typed is nonsense. Re-type it and be explicit. >> >>It was a bit brief, sorry about that. :-) >> >>To get a non-looping block to loop, you will need to put it in a loop or >>put a loop in it. >> >>In the first case, doing a 'break' in the block doesn't exit the loop. >>so you need to add an extra test for that. >> >>In the second case, doing a 'break' in the loop does exit the block, but >>finishes any code after the loop. So you may need an extra case in that >>case. >> >>Having a block that loops can simplify these conditions, in that a break >>alway exits the body of the block and stops the loop. A 'continue' can >>be used to skip the end of the block and start the next loop early. >> >>And you still have the option to put the block in a loop or loops in the >>block and they will work as they do now. >> >>I hope that clarifies what I was thinking a bit better. > > > > That is the long-standing nested loops 'issue', which is not going to be > solved here, nor should it be. We may not find a solution today, but where should it be addressed if not here? I don't really see the general issue of breaking out of loops as a problem, but was just addressing where it overlaps blocks and weather or not blocks should loop. > I am not sure that any solution to the issue will be sufficient for > everyone involved. That's the nature of programming in general isn't it. ;-) > The closest thing to a generic solution I can come > up with would be to allow for the labeling of for/while loops, and the > allowing of "break/continue ", which continues to that loop > (breaking all other loops currently nested within), or breaks that loop > (as well as all other loops currently nested within). > > Perhaps something like... > > while ... label 'foo': > for ... in ... label 'goo': > block ... label 'hoo': > if ...: > #equivalent to continue 'hoo' > continue > elif ...: > continue 'goo' > elif ...: > continue 'foo' > else: > break 'foo' > > Does this solve the nested loop problem? Yes. Do I like it? Not > really; three keywords in a single for/block statement is pretty awful. > On the upside, 'label' doesn't need to be a full-on keyword (it can be a > partial keyword like 'as' still seems to be). How about this for breaking out of all loops at once. class BreakLoop(Exception): """break out of nested loops""" try: for x in range(100): for y in range(100): for z in range(100): if x == 25 and y==72 and z==3: raise BreakLoop except BreakLoop: pass print 'x,y,z =', x,y,z Sometimes I would like a "try until :" for cases like this where you would use "except :pass". Cheers, Ron_Adam ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: > The whole PEP draft can be found here: > http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html I've updated this based on the feedback so far. The biggest change is that I've dropped the 'del' idea in favour of an optional 'finally' clause on for loops that finalises the iterator in addition to executing the code contained in the clause. I also added additional description of the purpose of user defined statements (factoring out exception handling boilerplate that is not easily factored into a separate function), and fixed the semantics so that __exit__() is called without an argument when the statement exits cleanly (previously, a template could not tell if the statement exited cleanly or not). I expanded on the generator section, indicating that the __exit__ method simply invokes next() if no exception is passed in (this makes the transaction example work correctly). I updated the auto_retry example to work with the new for loop finalisation approach, and added an example (reading the lines from multiple named files) where timely iterator finalisation is needed. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Paul Moore wrote: > On 5/8/05, Jp Calderone <[EMAIL PROTECTED]> wrote: > >> If such a construct is to be introduced, the ideal spelling would seem to >> be: >> >>for [VAR in] EXPR: >>BLOCK1 >>finally: >>BLOCK2 > > > While I have not been following this discussion at all (I don't have > the energy or time to follow the development of yet another proposal - > I'll wait for the PEP) this does read more naturally to me than any of > the other contortions I've seen passing by. Given this for loop syntax: for VAR in EXPR: BLOCK1 else: BLOCK2 finally: BLOCK3 And these semantics when a finally block is present: itr = iter(EXPR1) exhausted = False try: while True: try: VAR1 = itr.next() except StopIteration: exhausted = True break BLOCK1 if exhausted: BLOCK2 finally: try: BLOCK3 finally: itr_exit = getattr(itr, "__exit__", None) if itr_exit is not None: try: itr.__exit__(TerminateBlock) except TerminateBlock: pass "Loop on this iterator and finalise when done" would be written: for item in itr: process(item) finally: pass If you just want the finally clause, without finalising the iterator, you write it as you would now: try: for item in itr: process(item) finally: finalisation() I like it - I'll update the PEP redraft to use it instead of the 'del' idea. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
On 5/8/05, Jp Calderone <[EMAIL PROTECTED]> wrote: > If such a construct is to be introduced, the ideal spelling would seem to > be: > > for [VAR in] EXPR: > BLOCK1 > finally: > BLOCK2 While I have not been following this discussion at all (I don't have the energy or time to follow the development of yet another proposal - I'll wait for the PEP) this does read more naturally to me than any of the other contortions I've seen passing by. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam wrote: > Question: Is the 'for' in your case iterating over a sequence? or is it > testing for an assignment to determine if it should continue? Iterating over a sequence. If it's single-pass (and always single pass), you should use a user defined statement instead. > The difference is slight I admit, and both views can be said to be true > for 'for' loops iterating over lists also. But maybe looking at it as a > truth test of getting something instead of an iteration over a sequence > would fit better? When a variable to assign is not supplied then the > test would be of a private continue-stop variable in the iterator or a > StopIteration exception. No, that's the use case for user defined statements - if __enter__ raises TerminateBlock, then the body of the statement is not executed. What the for-loop part of the redrafted PEP is about is whether or not there should be an easy way to say "iterate over this iterator, and finalise it afterwards, regardless of how the iteration is ended", rather than having to use a try/finally block or a user defined statement for that purpose. I think I need to reorder those two sections - introduce user-defined statements first, then consider whether or not to add direct finalisation support to for loops. > If the keyword chosen is completely different from 'for' or 'while', > then it doesn't need a 'del' or 'finally' as that can be part of the new > definition of whatever keyword is chosen. That's the technique suggested for the single-pass user defined statements. However, a 'for loop with finalisation' is *still fundamentally an iterative loop*, and the syntax should reflect that. > So you might consider 'do', Guido responded with the following the other > day: [snip quote from Guido] > So it's not been ruled out, or followed though with, as far as I know. > And I think it will work for both looping and non looping situations. The same keyword cannot be used for the looping vs non-looping construct, because of the effect on the semantics of break and continue statements. The non-looping construct is the more fundamental of the two, since it can replace any current try/except/else/finally boilerplate, without any concern over whether or not the contained code using break or continue statements. A looping construct alters the meanings of those statements. >>The last option is to leave finalisation out of the 'for' loop syntax, and >>introduce a user defined statement to handle the finalisation: > > Yes, leaving it out of 'for' loop syntax is good. > > I don't have an opinion on user defined statements yet. But I think > they would be somewhat slower than a built in block that does the same > thing. What do you mean by 'built in block'? The user defined statements of the PEP redraft are simply a non-looping version of PEP 340's anonymous block statements. > Oops, meant that to say 'for-else' above ... > > The 'else' is new isn't it? I was thinking that putting a try-except > around the loop does the same thing as the else. Unless I misunderstand > it's use. No, the else clause on loops is a little known part of present day Python - it executes whenever the loop terminates naturally (i.e. not via a break statement). The only thing PEP 340 adds to for loops is the semantics to handle an argument to continue statements - it adds nothing to do with finalisation. My PEP redraft, on the other hand, suggests the introduction of a 'for loop with finalisation' that works fairly similarly to PEP 340's anonymous block statements. >>The PEP redraft already proposes a non-looping version as a new statement. >>However, since generators are likely to start using the new non-looping >>statement, it's important to be able to ensure timely finalisation of normal >>iterators as well. > > > Huh? I thought a normal iterator or generator doesn't need > finalization? If it does, then it's not normal. Has a word been coined > for iterators with try-finally's in them yet? An example was posted that looked like this: def all_lines(filenames): for name in filenames: stmt opening(name) as f: for line in f: yield line This is clearly intended for use as an iterator - it returns a bunch of lines. However, if the iterator is not finalised promptly, then the file that provided the last line may be left open indefinitely. By making such an iterator easy to write, it behooves the PEP to make it easy to use correctly. This need *can* be met by the 'consuming' user defined statement I posted earlier, but a more elegant solution is to be able to iterate over this generator normally, while also being able to ask Python to ensure the generator is finalised at the end of the iteration. Regards, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandl
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: > Eric Nieuwland <[EMAIL PROTECTED]> wrote: >> I don't know. Using 'del' in that place seems ackward to me. >> Why not use the following rule: >> for [VAR in] EXPR: >> SUITE >> If EXPR is an iterator, no finalisation is done. >> If EXPR is not an iterator, it is created at the start and destroyed >> at >> the end of the loop. > > You should know why that can't work. If I pass a list, is a list an > iterator? No, but it should neither be created nor destroyed before or > after. I suggested to create AN ITERATOR FOR THE LIST and destroy that at the end. The list itself remains untouched. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam <[EMAIL PROTECTED]> wrote: > > Josiah Carlson wrote: > >>I think a completely separate looping or non-looping construct would be > >>better for the finalization issue, and maybe can work with class's with > >>__exit__ as well as generators. > > > > From what I understand, the entire conversation has always stated that > > class-based finalized objects and generator-based finalized objects will > > both work, and that any proposal that works for one, but not the other, > > is not sufficient. > > That's good to hear. There seems to be some confusion as to weather or > not 'for's will do finalizing. So I was trying to stress I think > regular 'for' loops should not finalize. They should probably give an > error if an object with an try-finally in them or an __exit__ method. > I'm not sure what the current opinion on that is. But I didn't see it > in any of the PEPs. It's not a matter of 'will they be finalized', but instead a matter of 'will they be finalized in a timely manner'. From what I understand; upon garbage collection, any generator-based resource will be finalized via __exit__/next(exception)/... and any class-based resource will have its __del__ method called (as long as it is well-behaved), which can be used to call __exit__... > >>Having it loop has the advantage of making it break out in a better > >>behaved way. > > > > What you have just typed is nonsense. Re-type it and be explicit. > > It was a bit brief, sorry about that. :-) > > To get a non-looping block to loop, you will need to put it in a loop or > put a loop in it. > > In the first case, doing a 'break' in the block doesn't exit the loop. > so you need to add an extra test for that. > > In the second case, doing a 'break' in the loop does exit the block, but > finishes any code after the loop. So you may need an extra case in that > case. > > Having a block that loops can simplify these conditions, in that a break > alway exits the body of the block and stops the loop. A 'continue' can > be used to skip the end of the block and start the next loop early. > > And you still have the option to put the block in a loop or loops in the > block and they will work as they do now. > > I hope that clarifies what I was thinking a bit better. That is the long-standing nested loops 'issue', which is not going to be solved here, nor should it be. I am not sure that any solution to the issue will be sufficient for everyone involved. The closest thing to a generic solution I can come up with would be to allow for the labeling of for/while loops, and the allowing of "break/continue ", which continues to that loop (breaking all other loops currently nested within), or breaks that loop (as well as all other loops currently nested within). Perhaps something like... while ... label 'foo': for ... in ... label 'goo': block ... label 'hoo': if ...: #equivalent to continue 'hoo' continue elif ...: continue 'goo' elif ...: continue 'foo' else: break 'foo' Does this solve the nested loop problem? Yes. Do I like it? Not really; three keywords in a single for/block statement is pretty awful. On the upside, 'label' doesn't need to be a full-on keyword (it can be a partial keyword like 'as' still seems to be). Enough out of me, good night, - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: > For is already tuned to be as fast as possible, which makes sense; it is > used 4,523 times in Python 2.4.0's standard library, and easily hundreds > of thousands of times in user code. Changing the standard for loop is > not to be done lightly. Agreed! >>And why this isn't just as good: >> >> try: >> for value in iterator: >> BLOCK1 >> except StopIteration: >> BLOCK2 >> >>Is one extra line that bad? > > > I don't know what line you are referring to. Was referring to the 'try'., the 'except' would be in place of the else. Nick pointed out this wouldn't work as the 'for' already catches the StopIteration exception. >>I think a completely separate looping or non-looping construct would be >>better for the finalization issue, and maybe can work with class's with >>__exit__ as well as generators. > > From what I understand, the entire conversation has always stated that > class-based finalized objects and generator-based finalized objects will > both work, and that any proposal that works for one, but not the other, > is not sufficient. That's good to hear. There seems to be some confusion as to weather or not 'for's will do finalizing. So I was trying to stress I think regular 'for' loops should not finalize. They should probably give an error if an object with an try-finally in them or an __exit__ method. I'm not sure what the current opinion on that is. But I didn't see it in any of the PEPs. >>Having it loop has the advantage of making it break out in a better >>behaved way. > > What you have just typed is nonsense. Re-type it and be explicit. It was a bit brief, sorry about that. :-) To get a non-looping block to loop, you will need to put it in a loop or put a loop in it. In the first case, doing a 'break' in the block doesn't exit the loop. so you need to add an extra test for that. In the second case, doing a 'break' in the loop does exit the block, but finishes any code after the loop. So you may need an extra case in that case. Having a block that loops can simplify these conditions, in that a break alway exits the body of the block and stops the loop. A 'continue' can be used to skip the end of the block and start the next loop early. And you still have the option to put the block in a loop or loops in the block and they will work as they do now. I hope that clarifies what I was thinking a bit better. Ron_Adam ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote: > Ron Adam wrote: > >>I agree, re-using or extending 'for' doesn't seem like a good idea to me. > > > I agree that re-using a straight 'for' loop is out, due to performance and > compatibility issues with applying finalisation semantics to all such > iterative > loops (there's a reason the PEP redraft doesn't suggest this). > > However, it makes sense to me that a "for loop with finalisation" should > actually *be* a 'for' loop - just with some extra syntax to indicate that the > iterator is finalised at the end of the loop. Question: Is the 'for' in your case iterating over a sequence? or is it testing for an assignment to determine if it should continue? The difference is slight I admit, and both views can be said to be true for 'for' loops iterating over lists also. But maybe looking at it as a truth test of getting something instead of an iteration over a sequence would fit better? When a variable to assign is not supplied then the test would be of a private continue-stop variable in the iterator or a StopIteration exception. > However, as you say, 'del' isn't great for the purpose, but I was trying to > avoid introduding yet another keyword. I didn't say, that was Josiah, but I agree 'del' is not good. >An obvious alternative is to use > 'finally' instead: > >for [finally] [VAR in] EXPR: >BLOCK1 >else: >BLOCK2 > > It still doesn't read all that well, but at least the word more accurately > reflects the semantics involved. How about: [VAR from] EXPR: Could 'from' be reused in this context? If the keyword chosen is completely different from 'for' or 'while', then it doesn't need a 'del' or 'finally' as that can be part of the new definition of whatever keyword is chosen. I suggested reusing 'while' a few days ago because it fit the situation well, but come to the conclusion reusing either 'for' or 'while' should both be avoided. So you might consider 'do', Guido responded with the following the other day: #quote >[Greg Ewing] >> How about 'do'? >> >>do opening(filename) as f: >> ... >> >>do locking(obj): >> ... >> >>do carefully(): # :-) >> ... I've been thinking of that too. It's short, and in a nostalgic way conveys that it's a loop, without making it too obvious. (Those too young to get that should Google for do-loop. :-) I wonder how many folks call their action methods do() though. #endquote So it's not been ruled out, or followed though with, as far as I know. And I think it will work for both looping and non looping situations. > The last option is to leave finalisation out of the 'for' loop syntax, and > introduce a user defined statement to handle the finalisation: Yes, leaving it out of 'for' loop syntax is good. I don't have an opinion on user defined statements yet. But I think they would be somewhat slower than a built in block that does the same thing. Performance will be an issue because these things will be nested and possibly quite deeply. >>I wonder how much effect adding, 'for-next' and the 'StopIteration' >>exception check as proposed in PEP340, will have on 'for''s performance. > > I'm not sure what you mean here - 'for' loops already use a StopIteration > raised > by the iterator to indicate that the loop is complete. The code you posted > can't > work, since it also intercepts a StopIteration raised in the body of the loop. Oops, meant that to say 'for-else' above ... The 'else' is new isn't it? I was thinking that putting a try-except around the loop does the same thing as the else. Unless I misunderstand it's use. But you are right, it wouldn't work if the loop catches the StopIteration. >>I think a completely separate looping or non-looping construct would be >>better for the finalization issue, and maybe can work with class's with >>__exit__ as well as generators. > > > The PEP redraft already proposes a non-looping version as a new statement. > However, since generators are likely to start using the new non-looping > statement, it's important to be able to ensure timely finalisation of normal > iterators as well. Huh? I thought a normal iterator or generator doesn't need finalization? If it does, then it's not normal. Has a word been coined for iterators with try-finally's in them yet? Ron_Adam :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
On Sun, 08 May 2005 14:16:40 +1000, Nick Coghlan <[EMAIL PROTECTED]> wrote: >Ron Adam wrote: >> I agree, re-using or extending 'for' doesn't seem like a good idea to me. > >I agree that re-using a straight 'for' loop is out, due to performance and >compatibility issues with applying finalisation semantics to all such iterative >loops (there's a reason the PEP redraft doesn't suggest this). > >However, it makes sense to me that a "for loop with finalisation" should >actually *be* a 'for' loop - just with some extra syntax to indicate that the >iterator is finalised at the end of the loop. > >An option other than the one in my PEP draft would be to put 'del' at the end >of >the line instead of before EXPR: > > for [VAR in] EXPR [del]: > BLOCK1 > else: > BLOCK2 > >However, as you say, 'del' isn't great for the purpose, but I was trying to >avoid introduding yet another keyword. An obvious alternative is to use >'finally' instead: > > for [finally] [VAR in] EXPR: > BLOCK1 > else: > BLOCK2 > >It still doesn't read all that well, but at least the word more accurately >reflects the semantics involved. If such a construct is to be introduced, the ideal spelling would seem to be: for [VAR in] EXPR: BLOCK1 finally: BLOCK2 Jp ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam <[EMAIL PROTECTED]> wrote: > Josiah Carlson wrote: > > > You should know why that can't work. If I pass a list, is a list an > > iterator? No, but it should neither be created nor destroyed before or > > after. > > > > The discussion has been had in regards to why re-using 'for' is a > > non-starter; re-read the 200+ messages in the thread. > > > > - Josiah > > > I agree, re-using or extending 'for' doesn't seem like a good idea to me. Now that I've actually stopped to read Nick's PEP, my concern is that 'del', while being a keyword, would not be easy to spot embedded in the rest of the line, and a large number of these 'statements' will only be executed once, so the 'for' may confuse people. > I wonder how much effect adding, 'for-next' and the 'StopIteration' > exception check as proposed in PEP340, will have on 'for''s performance. For is already tuned to be as fast as possible, which makes sense; it is used 4,523 times in Python 2.4.0's standard library, and easily hundreds of thousands of times in user code. Changing the standard for loop is not to be done lightly. > And why this isn't just as good: > > try: > for value in iterator: > BLOCK1 > except StopIteration: > BLOCK2 > > Is one extra line that bad? I don't know what line you are referring to. > I think a completely separate looping or non-looping construct would be > better for the finalization issue, and maybe can work with class's with > __exit__ as well as generators. >From what I understand, the entire conversation has always stated that class-based finalized objects and generator-based finalized objects will both work, and that any proposal that works for one, but not the other, is not sufficient. > Having it loop has the advantage of making it break out in a better > behaved way. What you have just typed is nonsense. Re-type it and be explicit. > Hint: 'do' 'do' has been previously mentioned in the thread. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Ron Adam wrote: > I agree, re-using or extending 'for' doesn't seem like a good idea to me. I agree that re-using a straight 'for' loop is out, due to performance and compatibility issues with applying finalisation semantics to all such iterative loops (there's a reason the PEP redraft doesn't suggest this). However, it makes sense to me that a "for loop with finalisation" should actually *be* a 'for' loop - just with some extra syntax to indicate that the iterator is finalised at the end of the loop. An option other than the one in my PEP draft would be to put 'del' at the end of the line instead of before EXPR: for [VAR in] EXPR [del]: BLOCK1 else: BLOCK2 However, as you say, 'del' isn't great for the purpose, but I was trying to avoid introduding yet another keyword. An obvious alternative is to use 'finally' instead: for [finally] [VAR in] EXPR: BLOCK1 else: BLOCK2 It still doesn't read all that well, but at least the word more accurately reflects the semantics involved. If a new keyword is used to request iterator finalisation, it should probably include the word 'for' since it *is* a for loop: foreach [VAR in] EXPR: BLOCK1 else: BLOCK2 That is, a basic 'for' loop wouldn't finalise the iterator, but a 'foreach' loop would. The other difference is that the iterator in the 'foreach' loop has the chance to suppress exceptions other than TerminateBlock/StopIteration (by refusing to be finalised in response to the exception). The last option is to leave finalisation out of the 'for' loop syntax, and introduce a user defined statement to handle the finalisation: def consuming(iterable): itr = iter(iterable) try: yield itr finally: itr_exit = getattr(itr, "__exit__", None) if itr_exit is not None: try: itr_exit(TerminateBlock) except TerminateBlock: pass stmt consuming(iterable) as itr: for item in itr: process(item) With this approach, iterators can't swallow exceptions. This means that something like auto_retry() would once again have to be written as a class: class auto_retry(3, IOError): def __init__(self, times, exc=Exception): self.times = xrange(times-1) self.exc = exc self.succeeded = False def __iter__(self): attempt = self.attempt for i in self.times: yield attempt() if self.succeeded: break else: yield self.last_attempt() def attempt(self): try: yield self.succeeded = True except self.exc: pass def last_attempt(self): yield for attempt in auto_retry(3, IOError): stmt attempt: # Do something! # Including break to give up early # Or continue to try again without raising IOError > I wonder how much effect adding, 'for-next' and the 'StopIteration' > exception check as proposed in PEP340, will have on 'for''s performance. I'm not sure what you mean here - 'for' loops already use a StopIteration raised by the iterator to indicate that the loop is complete. The code you posted can't work, since it also intercepts a StopIteration raised in the body of the loop. > I think a completely separate looping or non-looping construct would be > better for the finalization issue, and maybe can work with class's with > __exit__ as well as generators. The PEP redraft already proposes a non-looping version as a new statement. However, since generators are likely to start using the new non-looping statement, it's important to be able to ensure timely finalisation of normal iterators as well. Tim and Greg's discussion the other day convinced me of this - that's why the idea of using 'del' to mark a finalised loop made its way into the draft. It can be done using a user defined statement (as shown above), but it would be nicer to have something built into the 'for' loop syntax to handle it. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Josiah Carlson wrote: > You should know why that can't work. If I pass a list, is a list an > iterator? No, but it should neither be created nor destroyed before or > after. > > The discussion has been had in regards to why re-using 'for' is a > non-starter; re-read the 200+ messages in the thread. > > - Josiah I agree, re-using or extending 'for' doesn't seem like a good idea to me. I wonder how much effect adding, 'for-next' and the 'StopIteration' exception check as proposed in PEP340, will have on 'for''s performance. And why this isn't just as good: try: for value in iterator: BLOCK1 except StopIteration: BLOCK2 Is one extra line that bad? I think a completely separate looping or non-looping construct would be better for the finalization issue, and maybe can work with class's with __exit__ as well as generators. Having it loop has the advantage of making it break out in a better behaved way. So may be Nicks PEP, would work better with a different keyword? Hint: 'do' Cheers, Ron_Adam ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Eric Nieuwland <[EMAIL PROTECTED]> wrote:
>
> Nick Coghlan wrote:
>
> > [...]
> > The whole PEP draft can be found here:
> > http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html
> > [...]
> > Used as follows::
> >
> > for del auto_retry(3, IOError):
> > f = urllib.urlopen("http://python.org/";)
> > print f.read()
>
> I don't know. Using 'del' in that place seems ackward to me.
> Why not use the following rule:
> for [VAR in] EXPR:
> SUITE
> If EXPR is an iterator, no finalisation is done.
> If EXPR is not an iterator, it is created at the start and destroyed at
> the end of the loop.
You should know why that can't work. If I pass a list, is a list an
iterator? No, but it should neither be created nor destroyed before or
after.
The discussion has been had in regards to why re-using 'for' is a
non-starter; re-read the 200+ messages in the thread.
- Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
Nick Coghlan wrote:
> [...]
> The whole PEP draft can be found here:
> http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html
> [...]
> Used as follows::
>
> for del auto_retry(3, IOError):
> f = urllib.urlopen("http://python.org/";)
> print f.read()
I don't know. Using 'del' in that place seems ackward to me.
Why not use the following rule:
for [VAR in] EXPR:
SUITE
If EXPR is an iterator, no finalisation is done.
If EXPR is not an iterator, it is created at the start and destroyed at
the end of the loop.
--eric
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 340: Deterministic Finalisation (new PEP draft, either a competitor or update to PEP 340)
PEP 340 contains several different ideas. This rewrite separates them into five
major areas:
- passing data into an iterator
- finalising iterators
- integrating finalisation into for loops
- the new non-looping finalising statement
- integrating all of these with generators.
The first area has nothing to do with finalisation, so it is not included in
this rewrite (Steven Bethard wrote an Enhanced Iterators pre-PEP which covers
only that area, though).
The whole PEP draft can be found here:
http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html
But I've inlined some examples that differ from or aren't in PEP 340 for those
that don't have time to read the whole thing (example numbers are from the PEP):
4. A template that tries something up to n times::
def auto_retry(n=3, exc=Exception):
for i in range(n):
try:
yield
except exc, err:
# perhaps log exception here
yield
raise # re-raise the exception we caught earlier
Used as follows::
for del auto_retry(3, IOError):
f = urllib.urlopen("http://python.org/";)
print f.read()
6. It is easy to write a regular class with the semantics of example 1::
class locking:
def __init__(self, lock):
self.lock = lock
def __enter__(self):
self.lock.acquire()
def __exit__(self, type, value=None, traceback=None):
self.lock.release()
if type is not None:
raise type, value, traceback
(This example is easily modified to implement the other examples; it shows that
generators are not always the simplest way to do things.)
8. Find the first file with a specific header::
for name in filenames:
stmt opening(name) as f:
if f.read(2) == 0xFEB0: break
9. Find the first item you can handle, holding a lock for the entire loop, or
just for each iteration::
stmt locking(lock):
for item in items:
if handle(item): break
for item in items:
stmt locking(lock):
if handle(item): break
10. Hold a lock while inside a generator, but release it when returning control
to the outer scope::
stmt locking(lock):
for item in items:
stmt unlocking(lock):
yield item
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
