Re: [Python-Dev] PEP 340 -- loose ends
Shane Holloway (IEEE) wrote: It might actually be workable in the transaction scenario, as well as others. I'm not sure if I love or hate the idea though. Given that this is officially a violation of the iterator protocol. . . (check the docs for well-behaved iterators) Another thing. In the specification of the Anonymous Block function, is there a reason that itr = EXPR1 instead of itr = iter(EXPR1)? It seems to be a dis-symmetry with the 'for' loop specification. Indeed - and a deliberate one, at least partly to discourage caching of block iterators. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
[ Guido ]: 1. Decide on a keyword to use, if any. Shouldn't be the other way around ? Decide to use *no* keyword, if that could be avoided. In my large inexperience *no keyword* is much better (if feasible): 1) No name conflicts with previous code: block, blocktemplate, whatever 2) ':' is already a block (broader sense) indication 3) Improved readbility: from PEP 340 def locking_opening(lock, filename, mode=r): block locking(lock): block opening(filename) as f: yield f from PEP 340 def locking_opening(lock, filename, mode=r): locking(lock): opening(filename) as f: yield f 4) Better to make the language parser more complex than the language exposed to end-users Following the PEP and this thread, it seems to me that __no keyword__ is less preferable than __some keyword__(=='block' so far), and I wonder why is not the reverse. Perhaps I missed something ? Besides, I think this solves many issues AOP was trying to tackle in a much cleaner, elegant -- therefore pythonic -- way. Outstanding. best regards, Senra -- Rodrigo Senra -- MSc Computer Engineerrodsenra(at)gpr.com.br GPr Sistemas Ltdahttp://www.gpr.com.br/ Personal Blog http://rodsenra.blogspot.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Reinhold Birkenfeld wrote: There is one problem with using no keyword: You cannot use arbitrary expressions in the new statement. [...] resource = opening(file.txt) resource: (...) The latter would have to be forbidden. Noam Raphael wrote: Can you explain why it would have to be forbidden please? Reinhold Birkenfeld wrote: Well, with it you could create suites with _any_ introducing identifier. Consider: with: (...) synchronized: (...) try: (...) transaction: (...) Do you understand my concern? It would be very, very hard to discern these user-defined statements from real language constructs. I think part of the debate is about whether that's good or bad. I happen to agree with you -- i think a keyword is necessary -- but i believe some people see an advantage in having the flexibility to make a real-looking construct. As i see it the argument boils down to: Python is not Lisp. There are good reasons why the language has keywords, why it distinguishes statements from expressions, uses indentation, and so on. All of these properties cause Python programs to be made of familiar and easily recognizable patterns instead of degenerating into a homogeneous pile of syntax. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Reinhold Birkenfeld wrote:
Noam Raphael wrote:
On 5/4/05, Reinhold Birkenfeld [EMAIL PROTECTED] wrote:
resource = opening(file.txt)
resource:
(...)
The latter would have to be forbidden.
Can you explain why it would have to be forbidden please?
Well, with it you could create suites with _any_ introducing
identifier. Consider:
[...]
transaction:
(...)
Do you understand my concern? It would be very, very hard to discern
these user-defined statements from real language constructs.
For each block statement, it is necessary to create a *new* iterator,
since iterators that have stopped are required to stay stopped. So at a
minimum, used-defined statements will need to call something, and thus
will have parentheses. The parentheses might be enough to make block
statements not look like built-in keywords.
PEP 340 seems to punish people for avoiding the parentheses:
transaction = begin_transaction()
transaction:
db.execute('insert 3 into mytable')
transaction:
db.execute('insert 4 into mytable')
I expect that only '3' would be inserted in mytable. The second use of
the transaction iterator will immediately raise StopIteration.
Shane
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Reinhold Birkenfeld wrote:
Shane Hathaway wrote:
For each block statement, it is necessary to create a *new* iterator,
Right.
since iterators that have stopped are required to stay stopped. So at a
minimum, used-defined statements will need to call something, and thus
will have parentheses. The parentheses might be enough to make block
statements not look like built-in keywords.
PEP 340 seems to punish people for avoiding the parentheses:
transaction = begin_transaction()
transaction:
db.execute('insert 3 into mytable')
transaction:
db.execute('insert 4 into mytable')
I expect that only '3' would be inserted in mytable. The second use of
the transaction iterator will immediately raise StopIteration.
Yes, but wouldn't you think that people would misunderstand it in this way?
Yes, they might. Just to be clear, the risk is that people will try to
write statements without parentheses and get burned because their code
doesn't get executed, right?
A possible workaround is to identify iterators that have already
finished. StopIteration doesn't distinguish between an iterator that
never yields any values from an iterator that has yielded all of its
values. Maybe there should be a subclass of StopIteration like
AlreadyStoppedIteration. Then, if a block statement gets an
AlreadyStoppedIteration exception from its iterator, it should convert
that to an error like InvalidBlockError.
Shane
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Shane Hathaway wrote: For each block statement, it is necessary to create a *new* iterator, since iterators that have stopped are required to stay stopped. So at a minimum, used-defined statements will need to call something, and thus will have parentheses. The parentheses might be enough to make block statements not look like built-in keywords. Definitely true for generators. Not necessarily true for iterators in general:: class Example(object): value = 0 result = False def __iter__(self): return self def next(self): self.result = not self.result if self.result: self.value += 1 return self.value else: raise StopIteration() :: e = Example() list(e) [1] list(e) [2] list(e) [3] It might actually be workable in the transaction scenario, as well as others. I'm not sure if I love or hate the idea though. Another thing. In the specification of the Anonymous Block function, is there a reason that itr = EXPR1 instead of itr = iter(EXPR1)? It seems to be a dis-symmetry with the 'for' loop specification. Thanks, -Shane (Holloway) ;) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
I'm forced by my day job to temporarily withdraw from the discussion about PEP 340 (I've used up my Python quota for the next several weeks). If agreement is reached in python-dev to suggest specific changes to the PEP, please let me know via mail sent directly to me and not cc'ed to python-dev. But please only if there is broad agreement on something. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Shane Hathaway wrote: For each block statement, it is necessary to create a *new* iterator, since iterators that have stopped are required to stay stopped. So at a minimum, used-defined statements will need to call something, and thus will have parentheses. Not necessarily! class Frobbing: def __neg__(self): begin_frobbing() try: yield finally: end_frobbing() frobbing = Frobbing() ... -frobbing: do_something() -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Shane Holloway (IEEE) wrote: Another thing. In the specification of the Anonymous Block function, is there a reason that itr = EXPR1 instead of itr = iter(EXPR1)? It seems to be a dis-symmetry with the 'for' loop specification. Hmm... yeah. That's strange. In fact, if it gets changed to itr = iter(EXPR1), as it probably ought to, all of the existing examples will continue to work. It will also be safe to start block iterators with a single variable, nullifying my argument about parentheses. So Reinhold's examples stand, except for the try block, since it clashes with a keyword. They read well, but when something goes wrong in the code, how would a new programmer crack these nuts? with: (...) synchronized: (...) transaction: (...) Thanks, -Shane (Holloway) ;) Once in a while I read a post by Shane Hway and start wondering when I wrote it and how I could've forgotten about it. And then I realize I didn't. :-) Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Greg Ewing wrote: Shane Hathaway wrote: For each block statement, it is necessary to create a *new* iterator, since iterators that have stopped are required to stay stopped. So at a minimum, used-defined statements will need to call something, and thus will have parentheses. Not necessarily! class Frobbing: def __neg__(self): begin_frobbing() try: yield finally: end_frobbing() frobbing = Frobbing() ... -frobbing: do_something() Larry Wall would hire you in a heartbeat. ;-) Maybe there's really no way to prevent people from writing cute but obscure block statements. A keyword like block or suite would give the reader something firm to hold on to. Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Phillip J. Eby wrote: Specifically, I propose that PEP 340 *not* allow the use of normal iterators. Instead, the __next__ and __exit__ methods would be an unrelated protocol. This would eliminate the need for a 'next()' builtin, and avoid any confusion between today's iterators and a template function for use with blocks. I would extend this to say that invoking the blocktemplate decorator should eliminate the conventional iteration interface, preventing the following problematically silent bug: for l in synchronized(mylock): # This lock is not released promptly! break My argument is that this is both Explicit (i.e., better than implicit) and One Obvious Way (because using existing iterators just Another Way to do a for loop). It also doesn't allow Errors (using an iterator with no special semantics) to Pass Silently. While I agree these are advantages, a bigger issue for me would be the one above: keeping a block template which expects prompt finalisation from being inadvertently used in a conventional for loop which won't finalise on early termination of the loop. I'd also suggest that the blocktemplate decorator accept any iterator, not just generators. Of course, since Practicality Beats Purity, I could give this all up. But I don't think the Implementation is Hard to Explain, as it should be just as easy as Guido's proposal. I think it would be marginally easier to explain, since the confusion between iterators and block templates would be less of a distraction. Really, the only thing that changes is that you get a TypeError when a template function returns an iterator instead of a block template, and you have to use the decorator on your generators to explicitly label them safe for use with blocks. I'd add raising a TypeError when a block template is passed to the iter() builtin to the list of differences from the current incarnation of the PEP. As for Phillip, I think using different API's is a good way to more clearly emphasise the difference in purpose between conventional for loops and the new block statement, but I'm also a little concerned about incorrectly passing a block template to a for loop. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Guido van Rossum wrote: 1. I still can't decide on keyword vs. no keyword, but if we're going to have a keyword, I haven't seen a better proposal than block. So it's either block or nothing. I'll sleep on this. Feel free to start an all-out flame war on this in c.l.py. ;-) I quite like 'block', but can live with no keyword (since it then becomes a practical equivalent to user-defined statements). 2. No else clause; the use case is really weak and there are too many possible semantics. It's not clear whether to generalize from for/else, or if/else, or what else. Agreed. The order I posted my list of semantic options was the order I thought of them, but I ended up agreeing with the votes Aahz posted. 3. I'm leaning against Phillip's proposal; IMO it adds more complexity for very little benefit. See my response to Phillip. I think there could be an advantage to it if it means that for l in synchronized(lock) raises an immediate error instead of silently doing the wrong thing. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Nick Coghlan a écrit : 3. I'm leaning against Phillip's proposal; IMO it adds more complexity for very little benefit. See my response to Phillip. I think there could be an advantage to it if it means that for l in synchronized(lock) raises an immediate error instead of silently doing the wrong thing. First, I really think this PEP is needed for Python. But this is express exactly my main concern about it ! As far as I understand it, iterator-for-blocks and iterator-for-loops are two different beasts. Even if iterator-for-loops can be used within a block without damage, the use of iterator-for-block in a loop can lead to completely unpredictable result (and result really hard to find since they'll possibly involve race conditions or dead locks). To try being as clear as possible, I would say the iterator-for-loops are simplified iterator-for-blocks. IOW, if I were to put them in a class inheritance hierarchy (I don't say we should put them into one ;) ) iterator-for-block would be the base class of iterator-for-loop. Thus, as for-loops require an iterator-for-loop, they would raise an error if used with an iterator-for-block. But as blocks require an iterator-for-blocks they will allow iterator-for-loops too ! Cheers, Pierre -- Pierre Barbier de Reuille INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP Botanique et Bio-informatique de l'Architecture des Plantes TA40/PSII, Boulevard de la Lironde 34398 MONTPELLIER CEDEX 5, France tel : (33) 4 67 61 65 77fax : (33) 4 67 61 56 68 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Pierre Barbier de Reuille wrote: Even if iterator-for-loops can be used within a block without damage, the use of iterator-for-block in a loop can lead to completely unpredictable result (and result really hard to find since they'll possibly involve race conditions or dead locks). I had a longish post written before I realised I'd completely misunderstood your comment. You were actually agreeing with me, so most of my post was totally beside the point. Anyway, to summarise the argument in favour of separate API's for iterators and block templates, the first code example below is a harmless quirk (albeit an irritating violation of TOOWTDI). The second and third examples are potentially serious bugs: block range(10) as i: # Just a silly way to write for i in range(10) for f in opening(name): # When f gets closed is Python implementation dependent for lock in synchronized(mylock): # When lock gets released is Python implementation dependent Cheers, Nick. P.S. Dear lord, synchronized is an aggravating name for that function. I keep wanting to spell it with a second letter 's', like any civilised person ;) -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Nick Coghlan [EMAIL PROTECTED] writes: Paul Svensson wrote: On Tue, 3 May 2005, Nick Coghlan wrote: I'd also suggest that the blocktemplate decorator accept any iterator, not just generators. So you want decorators on classes now ? A decorator is just a function - it doesn't *need* to be used with decorator syntax. I just think the following code should work for any iterator: block blocktemplate(itr): # Do stuff But in @blocktemplate def foo(...): ... blocktemplate isn't passed an iterator, it's passed a callable that returns an iterator. Cheers, mwh -- . - the pointyour article - . |- a long way | -- Christophe Rhodes, ucam.chat ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Hi Guido, On Mon, May 02, 2005 at 17:55 -0700, Guido van Rossum wrote: These are the loose ends on the PEP (apart from filling in some missing sections): 1. Decide on a keyword to use, if any. I just read the PEP340 basically the first time so bear with me. First i note that introducing a keyword 'block' would break lots of programs, among it half of PyPy. Unlike many other keywords 'block' is a pretty common variable name. For invoking blocktemplates i like the no-keyword approach, instead. However, i would find it much clearer if *defining* blocktemplates used a new keyword, like: blocktemplate opening(filename, mode=r): ... because this immediately tells me what the purpose and semantics of the folowing definition is. The original overloading of 'def' to mean generators if the body contains a yield statement was already a matter of discussion (ASFAIK). When i came to Python it was at 2.2 and i remember wondering about this def oddity. Extending poor old 'def' functions now to possibly mean block templates gives me semantical overload even if it is justified from an implementation point of view. I am talking purely about (my sense of) code readability here not about implementation. cheers, holger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
[Holger] 1. Decide on a keyword to use, if any. I just read the PEP340 basically the first time so bear with me. Thanks for reviewing! First i note that introducing a keyword 'block' would break lots of programs, among it half of PyPy. Unlike many other keywords 'block' is a pretty common variable name. For invoking blocktemplates i like the no-keyword approach, instead. Good point (the code from Queue.py quoted by Jim Jewett also uses block as a variable name :-). There has been much argument on both sides. I guess we may need to have a subcommittee to select the keyword (if any) ... Maybe if we can't go without a keyword, 'with' would be okay after all; I'm not so strongly in favor of a Pascal/VB-style with-statement after reading the C# developers' comments (see reference in the PEP). However, i would find it much clearer if *defining* blocktemplates used a new keyword, like: blocktemplate opening(filename, mode=r): ... because this immediately tells me what the purpose and semantics of the folowing definition is. The original overloading of 'def' to mean generators if the body contains a yield statement was already a matter of discussion (ASFAIK). When i came to Python it was at 2.2 and i remember wondering about this def oddity. Extending poor old 'def' functions now to possibly mean block templates gives me semantical overload even if it is justified from an implementation point of view. I am talking purely about (my sense of) code readability here not about implementation. Hm... Maybe you also want to have separate function and procedure keywords? Or static typing? 'def' can be used to define all sorts of things, that is Python's beauty! -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
At 05:55 PM 5/2/05 -0700, Guido van Rossum wrote: 3. I'm leaning against Phillip's proposal; IMO it adds more complexity for very little benefit. Little benefit, I'll agree with, even though there are EIBTI and TOOWTDI benefits as well as Errors Should Never Pass Silently. But the only added implementation complexity is the decorator -- balanced against the removal of the need for a 'next()' builtin. I also believe that the approach actually *reduces* pedagogical complexity by not allowing any blurring between the concept of an iterator and the concept of a block template. Since I'm not sure if anybody besides you is aware of what I proposed, I'll attempt to recap here, and then step to allow discussion. If there's no community support, I'll let it die a natural death, because it's ultimately a purity question rather than a practical one, though I think that other people who teach Python programming should weigh in on this. Specifically, I propose that PEP 340 *not* allow the use of normal iterators. Instead, the __next__ and __exit__ methods would be an unrelated protocol. This would eliminate the need for a 'next()' builtin, and avoid any confusion between today's iterators and a template function for use with blocks. Because today's generators were also not written with blocks in mind, it would also be necessary to use a @decorator to declare that a generator is in fact a block template. Possibly something like: @blocktemplate def retry(times): for i in xrange(times): try: yield except StopIteration: return except: continue else: return raise My argument is that this is both Explicit (i.e., better than implicit) and One Obvious Way (because using existing iterators just Another Way to do a for loop). It also doesn't allow Errors (using an iterator with no special semantics) to Pass Silently. Of course, since Practicality Beats Purity, I could give this all up. But I don't think the Implementation is Hard to Explain, as it should be just as easy as Guido's proposal. Instead of a 'next()' builtin, one would instead implement a 'blocktemplate' decorator (or whatever it's to be called). The same __next__/__exit__/next methods have to be implemented as in Guido's proposal. Really, the only thing that changes is that you get a TypeError when a template function returns an iterator instead of a block template, and you have to use the decorator on your generators to explicitly label them safe for use with blocks. (Hand-crafted block templates still just implement __next__ and __exit__, in the same way as they would under Guido's proposal, so no real change there.) Guido may also have other reasons to take a different direction that he may not have expressed; e.g. maybe in Py3K there'll be no for, just iter(x) as y:? Or...? I don't claim to have any special smarts about this, but other people (including Guido) have previously expressed reservations about the near-blending of iteration and block control that PEP 340 allows. So, I've thrown out this proposal as an attempt to address those reservations. YMMV. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
At 09:39 PM 5/2/05 -0400, Phillip J. Eby wrote: attempt to recap here, and then step to allow discussion. If there's no Argh. That was supposed to be, step aside to allow discussion. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
Phillip J. Eby wrote: Specifically, I propose that PEP 340 *not* allow the use of normal iterators. Instead, the __next__ and __exit__ methods would be an unrelated protocol. This would eliminate the need for a 'next()' builtin, and avoid any confusion between today's iterators and a template function for use with blocks. PEP 340 does not address normal iterators very well, but a properly-constructed iterator will behave correctly. The PEP though is very generator-focussed. The issues I see for normal iterators (and that need to be addressed/stated in the PEP) are: 1. No automatic handling of parameters passed to __next__ and __exit__. In a generator, these will raise at the yield-statement or -expression. A normal iterator will have to take care of this manually. This could be an argument to only allow generator-iterators to be used with PEP 340 semantics (i.e. continue EXPR, block), but I don't think it's a very compelling one. Although perhaps the initial implementation could be restricted to generator-iterators. So if a for-loop used `continue EXPR` it would have a check (at the start of the for loop) that the iterator is a generator-iterator. Likewise, a block-statement would always include this check. As another option, it might be worthwhile creating a base iterator type with correct semantics. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
[Delaney, Timothy] PEP 340 does not address normal iterators very well, but a properly-constructed iterator will behave correctly. This is by design. The PEP though is very generator-focussed. Disagree. The PEP describes most everything (e.g. the block statement semantics) in terms of iterators, and then describes how the new APIs behave for generators. The issues I see for normal iterators (and that need to be addressed/stated in the PEP) are: 1. No automatic handling of parameters passed to __next__ and __exit__. In a generator, these will raise at the yield-statement or -expression. A normal iterator will have to take care of this manually. Not sure what you mean by this. If __next__() is defined, it is passed the parameter; if only next() is defined, a parameter (except None) is an error. That seems exactly right. Also, if __exit__() isn't defined, the exception is raised, which is a very sensible default behavior (and also what will happen to a generator that doesn't catch the exception). This could be an argument to only allow generator-iterators to be used with PEP 340 semantics (i.e. continue EXPR, block), but I don't think it's a very compelling one. Neither do I. :-) Although perhaps the initial implementation could be restricted to generator-iterators. So if a for-loop used `continue EXPR` it would have a check (at the start of the for loop) that the iterator is a generator-iterator. Likewise, a block-statement would always include this check. But what would this buy you except an arbitrary restriction? As another option, it might be worthwhile creating a base iterator type with correct semantics. Well, what would the correct semantics be? What would passing a parameter to a list iterator's __next__() method mean? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 -- loose ends
[Delaney, Timothy] What I meant is that there are no examples of how to actually implement the correct semantics for a normal iterator. Doing it right is non-trivial, especially with the __next__ and __exit__ interaction (see below). Depends on what you mean by right. Ignoring the argument to __next__() and not implementing __exit__() seems totally right to me. [...] What I meant is how the iterator is meant to handle the parameters passed to each method. PEP 340 deals with this by stating that exceptions will be raised at the next yield-statement or -expression. I think we need an example though of how this would translate to a normal iterator. Something along the lines of:: class iterator (object): def next (self): return self.__next__() def __next__(self, arg=None): value = None if isinstance(arg, ContinueIteration): Oops. Read the most recent version of the PEP again. __next__() doesn't take an exception argument, it only takes a value. Maybe this removes your concern? value = arg.value elif arg is not None: raise arg if value is None: raise StopIteration return value That's a very strange iterator; it immediately terminates unless you call __next__() with a non-None argument, then it returns the argument value. I'm having a hard time understanding what you meant to say. Also note that the very *first* call to __next__() is not supposed to have an argument. The argument (normally) only comes from continue EXPR and that can only be reached after the first call to __next__(). This is exactly right for generators -- the first __next__() call there starts the generator at the top of its function body, executing until the first yield is reached. def __exit__(self, type=None, value=None, traceback=None): if (type is None) and (value is None) and (traceback is None): type, value, traceback = sys.exc_info() You shouldn't need to check for traceback is None. Also, even though the PEP suggests that you can do this, I don't see a use case for it -- the translation of a block-statement never calls __exit__() without an exception. if type is not None: try: raise type, value, traceback except type, exc: return self.__next__(exc) return self.__next__() Ah, here we see the other misconception (caused by not reading the most recent version of the PEP). __exit__() shouldn't call __next__() -- it should just raise the exception passed in unless it has something special to do. Let me clarify all this with an example showing how you could write synchronized() as an iterator instead of a generator. class synchronized: def __init__(self, lock): self.lock = lock self.state = 0 def __next__(self, arg=None): # ignores arg if self.state: assert self.state == 1 self.lock.release() self.state += 1 raise StopIteration else: self.lock.acquire() self.state += 1 return None def __exit__(self, type, value=None, traceback=None): assert self.state in (0, 1, 2) if self.state == 1: self.lock.release() raise type, value, traceback As another option, it might be worthwhile creating a base iterator type with correct semantics. Well, what would the correct semantics be? What would passing a parameter to a list iterator's __next__() method mean? Sorry - I meant for user-defined iterators. And the correct semantics would be something like the example above I think. Except that I think most of it would need to be in a separate method (e.g. _next) for base classes to call - then things would change to be something like:: class iterator (object): ... def _next (self, arg): if isinstance(arg, ContinueIteration): return arg.value elif arg is not None: raise arg def __next__(self, arg=None): value = self._next(arg) if value is None: raise StopIteration return value ... I think this is all based on a misunderstanding of the PEP. Also, you really don't need to implement __exit__() unless you have some cleanup to do -- the default behavior of the block translation only calls it if defined, and otherwise simply raises the exception. Finally, I think there is another loose end that hasn't been addressed:: When __next__() is called with an argument that is not None, the yield-expression that it resumes will return the value attribute of the argument. If it resumes a yield-statement, the value is ignored (or should this be considered an error?). When the
Re: [Python-Dev] PEP 340 -- loose ends
Guido van Rossum wrote: Oops. Read the most recent version of the PEP again. __next__() doesn't take an exception argument, it only takes a value. Maybe this removes your concern? Actually, I misinterpreted it, assuming that the value passed in was an exception instance because the previous versions worked that way. This has been going on too long ;) Ah, here we see the other misconception (caused by not reading the most recent version of the PEP). __exit__() shouldn't call __next__() -- it should just raise the exception passed in unless it has something special to do. Ah - I think this needs to be explained better. In particular, in the specification of the __next__ and __exit__ methods it should state what exceptions are expected to be raised under what circumstances - in particular, that __exit__ is expected to raise the passed in exception or StopIteration. This is only explained in the Generator Exception Handling specification, but it's applicable to all iterators. Finally, I think there is another loose end that hasn't been addressed:: When __next__() is called with an argument that is not None, the yield-expression that it resumes will return the value attribute of the argument. If it resumes a yield-statement, the value is ignored (or should this be considered an error?). When the *initial* call to __next__() receives an argument that is not None, the generator's execution is started normally; the argument's value attribute is ignored (or should this be considered an error?). When __next__() is called without an argument or with None as argument, and a yield-expression is resumed, the yield-expression returns None. Good catch. My opinion is that each of these should be an error. Personally, I think not using the value passed into __next__() should not be an error; that's about the same as not using the value returned by a function you call. Now that I understand that the parameter to __next__ is not an exception, I agree. I agree that calling the initial __next__() of a generator with a non-None argument should be considered an error; this is likely caused by some kind of logic error; it can never happen when the generator is called by a block statement. Cheers. Tim Delaney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
