Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Jim Jewett wrote: The only members that need special attention are (f_code, f_lasti) and possibly (f_blockstack, f_iblock). You don't even need to take care of f_code. The thunk and its surrounding function can share the same code. The thunk gets compiled into the function the same way the body of a for loop would. (f_code, f_lasti) would need to be replaced with a stack of pairs. Finishing a code string would mean popping this stack, rather than popping the whole frame. There doesn't need to be a stack; each thunk can store its own f_lasti. One also needs to store f_back, and, to avoid exception weirdness, f_exc_XXX. In this way, calling the thunk is much like resuming a generator. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Guido van Rossum wrote: Even without a block-statement, these two changes make yield look a lot like invoking a thunk -- but it's more efficient, since calling yield doesn't create a frame. I like PEP 340 a lot, probably as much or more than any thunk ideas I've seen. But I want to defend thunks here a little. It is possible to implement thunks without them creating their own frame. They can reuse the frame of the surrounding function. So a new frame does not need to be created when the thunk is called, and, much like with a yield statement, the frame is not taken down when the thunk completes running. The implementation just needs to take care to save and restore members of the frame that get clobbered when the thunk is running. Cells would of course not be required if the thunk does not create its own frame. The main advantage of thunks that I can see is that you can save the thunk for later, like a callback for a button widget (the thunk then becomes a closure). You can't use a yield-based block for that (except in Ruby, which uses yield syntax with a thunk-based implementation). But I have to say that I almost see this as an advantage: I think I'd be slightly uncomfortable seeing a block and not knowing whether it will be executed in the normal control flow or later. Defining an explicit nested function for that purpose doesn't have this problem for me, because I already know that the 'def' keyword means its body is executed later. I would also be uncomfortable if the thunk could be called at a later time. This can be disallowed by raising an exception if such an attempt is made. Such a restriction would not be completely arbitrary. One consequence of having the thunk borrow its surrounding function's frame is that it does not make much sense, implementationally speaking, to allow the thunk to be called at a later time (although I do realize that it's difficult to implement is not a good argument for anything). The other problem with thunks is that once we think of them as the anonymous functions they are, we're pretty much forced to say that a return statement in a thunk returns from the thunk rather than from the containing function. Doing it any other way would cause major weirdness when the thunk were to survive its containing function as a closure (perhaps continuations would help, but I'm not about to go there :-). If it is accepted that the thunk won't be callable at a later time, then I think it would seem normal that a return statement would return from the surrounding function. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Greg Ewing wrote: I also have a thought concerning whether the block argument to the function should come first or last or whatever. My solution is that the function should take exactly *one* argument, which is the block. Any other arguments are dealt with by currying. In other words, with_file above would be defined as def with_file(filename): def func(block): f = open(filename) try: block(f) finally: f.close() return func This would also make implementation much easier. The parser isn't going to know that it's dealing with anything other than a normal expression statement until it gets to the 'as' or ':', by which time going back and radically re-interpreting a previous function call could be awkward. I made an example implementation, and this wasn't an issue. It took some code to stick the thunk into the argument list, but it was pretty straightforward. The syntax that is actually used by the parser can be the same regardless of whether or not argument list augmentation is done, so the parser will not find one more awkward than the other. This way, the syntax is just expr ['as' assignment_target] ':' suite and the expr is evaluated quite normally. Requiring arguments other than the block to be dealt with by currying can lead to problems. I won't claim these problems are serious, but they will be annoying. Say, for example, you create a block-accepting function that takes no arguments. Naturally, you would define it like this: def f(block): do_something_with_block Now, say you want to add to this function an optional argument, so you wrap another function around it like in your 'with_file' example above. Unfortunately, now you need to go find every call of this function and add empty parentheses. This is annoying. Remember the first time you added optional arguments to a function and what a relief it was not to have to go find every call to that function and stick in the extra argument? Those days are over! (well, in this case anyway.) Some people, aware of this problem of adding optional arguments, will define *all* of their block-accepting functions so that they are wrapped in another function, even if that function takes no arguments (and wars, annoying ones, will be fought over whether this is the right way to do it or not!): def f(): def real_func(block): pass return real_func Now the documentation gets confusing. Just saying that the function doesn't take any non-block arguments isn't enough. You would need very specific language, which many library authors will not provide. And there will always be that extra step in thought: do I need the stupid parentheses or not? There will inevitably be people (including me) who get the parentheses wrong because of absentmindedness or carelessness. This will be an extra little speed bump. Now, you may say that all these arguments apply to function decorators, so why have none of these problems appeared? The difference is that defining a function takes a long time, so a little speed bump when decorating it isn't a big deal. But blocks can be defined almost instantly. Much of their purpose has to do with making things quicker. Speed bumps are therefore a bigger deal. This will also be an issue for beginners who use python. A beginner won't necessarily have a good understanding of a function that returns a function. But such an understanding would be required simply to *use* block-accepting functions. Otherwise it would be completely mysterious why sometimes one sees this f(a,b,c) as i: pass and sometimes this g as i: pass even though both of these cases just seem to call the function that appears next to 'as' (imagine you don't have the source of 'f' and 'g'). Even worse, imagine finally learning the rule that parentheses are not allowed if there are zero arguments, and then seeing: h() as i: pass Now it would just seem arbitrary whether or not parentheses are required or disallowed. Such an issue may seem trivial to an experienced programmer, but can be very off-putting for a beginner. Another set of question arose for me when Barry started musing over the combination of blocks and decorators. What are blocks? Well, obviously they are callable. What do they return? The local namespace they created/modified? I think the return value of a block should be None. In constructs like with_file, the block is being used for its side effect, not to compute a value for consumption by the block function. I don't see a great need for blocks to be able to return values. If you google filetype:rb yield, you can see many the uses of yield in ruby. By looking for the uses in which yield's return value is used, you can find blocks that return values. For example, t = yield() or unless yield() indicate that a block is returning a value. It is true that most of the time blocks do not return values, but
Re: [Python-Dev] anonymous blocks
Shannon -jj Behrens wrote:
Have you guys considered the following syntax for anonymous blocks? I
think it's possible to parse given Python's existing syntax:
items.doFoo(
def (a, b) {
return a + b
},
def (c, d) {
return c + d
}
)
There was a proposal in the last few days on comp.lang.python that allows
you to do this in a way that requires less drastic changes to python's
syntax. See the thread pre-PEP: Suite-Based Keywords (shamless plug)
(an earlier, similar proposal is here:
http://groups.google.co.uk/groups?selm=mailman.403.1105274631.22381.python-list
%40python.org ).
In short, if doFoo is defined like:
def doFoo(func1, func2):
pass
You would be able to call it like:
doFoo(**):
def func1(a, b):
return a + b
def func2(c, d):
return c + d
That is, a suite can be used to define keyword arguments.
-Brian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Guido van Rossum wrote: See the thread pre-PEP: Suite-Based Keywords (shamless plug) (an earlier, similar proposal is here: http://groups.google.co.uk/groups?selm=mailman.403.1105274631.22381.python-list %40python.org ). In short, if doFoo is defined like: def doFoo(func1, func2): pass You would be able to call it like: doFoo(**): def func1(a, b): return a + b def func2(c, d): return c + d That is, a suite can be used to define keyword arguments. I'm still not sure how this is particularly solving a pressing problem that isn't solved by putting the function definitions in front of the call. I saw the first version of the proto-PEP and didn't think that the motivating example (keeping the getx/setx methods passed to a property definition out of the class namespace) was all that valuable. OK. I think most people (myself included) who would prefer to define properties (and event handlers, etc.) in this way are motivated by the perception that the current method is just ugly. I don't know that it solves any pressing problems. Two more issues: (1) It seems that *every* name introduced in the block automatically becomes a keyword argument. This looks like a problem, since you could easily need temporary variables there. (I don't see that a problem with class bodies because the typical use there is only method and property definitions and the occasional instance variable default.) Combining the suite-based keywords proposal with the earlier, 'where' proposal (linked in my above post), you would be able to name variables individually in the case that temporary variables are needed: f(x=x): x = [i**2 for i in [1,2,3]] (2) This seems to be attaching a block to a specific function call but there are more general cases: e.g. you might want to assign the return value of doFoo() to a variable, or you might want to pass it as an argument to another call. The 'where' proposal also doesn't have this problem. Any expression is allowed. *If* we're going to create syntax for anonymous blocks, I think the primary use case ought to be cleanup operations to replace try/finally blocks for locking and similar things. I'd love to have syntactical support so I can write blahblah(myLock): code code code instead of myLock.acquire() try: code code code finally: myLock.release() Well, that was my other proposal, pre-PEP: Simple Thunks (there is also an implementation). It didn't seem to go over all that well. I am going to try to rewrite it and give more motivation and explanation (and maybe use 'with' and 'from' instead of 'do' and 'in' as keywords). -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] anonymous blocks
Guido van Rossum wrote: @acquire(myLock): code code code It would certainly solve the problem of which keyword to use! :-) And I think the syntax isn't even ambiguous -- the trailing colon distinguishes this from the function decorator syntax. I guess it would morph '@xxx' into user-defined-keyword. How would acquire be defined? I guess it could be this, returning a function that takes a callable as an argument just like other decorators: def acquire(aLock): def acquirer(block): aLock.acquire() try: block() finally: aLock.release() return acquirer and the substitution of @EXPR: CODE would become something like def __block(): CODE EXPR(__block) Why not have the block automatically be inserted into acquire's argument list? It would probably get annoying to have to define inner functions like that every time one simply wants to use arguments. For example: def acquire(block, aLock): aLock.acquire() try: block() finally: aLock.release() @acquire(myLock): code code code Of course, augmenting the argument list in that way would be different than the behavior of decorators as they are now. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: Re: anonymous blocks
Fredrik Lundh wrote: Brian Sabbey wrote: If suites were commonly used as above to define properties, event handlers and other callbacks, then I think most people would be able to comprehend what the first example above is doing much more quickly than the second. wonderful logic, there. good luck with your future adventures in language design. /F I'm just trying to help python improve. Maybe I'm not doing a very good job, I don't know. Either way, there's no need to be rude. If I've broken some sort of unspoken code of behavior for this list, then maybe it would be easier if you just 'spoke' it (perhaps in a private email or in the description of this list on python.org). I'm not sure what your point is exactly. Are you saying that any language feature that needs to be commonly used to be comprehendible will never be comprehendible because it will never be commonly used? If so, then I do not think you have a valid point. I never claimed that keyword suites *need* to be commonly used to be comprehendible. I only said that if they were commonly used they would be more comprehendible than the alternative. I happen to also believe that seeing them once or twice is enough to make them about equally as comprehendible as the alternative. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] thunks (was: code blocks using 'for' loops and generators)
Jim Jewett wrote: It may be time to PEP (or re-PEP), if only to clarify what people are actually asking for. I will PEPify this, unless someone does not think I am the correct person to do so. The PEP is probably a better place to try to address questions you raise, as well as give the rationale that Josiah Carlson was looking for. But, in short: Brian Sabbey's example from message http://mail.python.org/pipermail/python-dev/2005-March/052202.html *seems* reasonably clear, but I don't see how it relates in any way to for loops or generators, except as one (but not the only) use case. The original post in this thread was an idea about using 'for' loops and generators, but that idea has since been replaced with something else. (1) Calls for Ruby blocks or thunks are basically calls for placeholders in a function. These placeholders will be filled with code from someplace else, but will execute in the function's own local namespace. It wasn't my intention that the thunk would execute in the function's namespace (function here is to mean the function that takes the thunk as an argument). I was thinking that scope rules for the thunk would mimic the rules for control flow structures. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] code blocks using 'for' loops and generators
Samuele Pedroni wrote: My point is that a suite-based syntax can only be a half substitute for lambda and anyway requiring a suite seems overkill and unnatural for the just 1 expression case, for example predicates. IOW a suite-based syntax is not a lambda killer in itself, I would not try to stress that point. I see your point (also I see Greg Ewing's related point). multiple dispatch has nothing to do with syntax, in fact usual call syntax is sufficient, and people do use multiple dispatch sometimes, and decorators now can be even used to sugar up the definition side of it. But one needs to use decorators or some other mechanism for the sugar, that is all I intended the phrase does not give syntactic support to mean. Perhaps syntactic sugar is the correct term to have used. for something that would be rarely used, I do not think well that's up to discussion to discover ok well, but this is stated without even trying to come up with a syntax for that case. Notice that the first time around Guido himself would have preferred if achievable a multithunk syntax, he obviously can have changed his mind. But, yes, syntax vs expressivity is the key issue here. Ok. Allow me to try. Up to a choice of (or existence of!) keywords, the simplest to me is: def add(thunk1, thunk2, other): print thunk1(1,2) + thunk2(3,4) + other with x,y from add(100): value x*y also a,b: # yikes?? value a*b # this is thunk2 -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] code blocks using 'for' loops and generators
be guaranteed to run under all conditions, I think it would be useful if it could be arranged so that for x in somegenerator(): ... raise Blather ... would caused any finallies that the generator was suspended inside to be executed. Then the semantics would be the same as if the for-loop-over-generator were implemented by passing a thunk to a function that calls it repeatedly. One difficulty is that one can never know if the user intends to still use the generator, like so: a = somegenerator() try: for x in a: raise Blather except: a.next() I think they only way you can really be sure .next() will not be called again is if the generator is no longer referenced. Someone submitted a patch once to execute finallies when the generator is __del__eted, but it was rejected for various reasons. In my original post in this thread I tried to provide a mechanism such as you describe by providing __call__ as an alternative to 'next', but now I am convinced that it is better to introduce a new syntax instead of re-using generators. Incidentally, passing the thunk behind the scenes as the first argument (as mentioned previously) allows one to avoid using lambda to do things such as sort (I hear lambdas are on the way out), while remaining anonymous: with x, y from a.sort(): value cmp(x.k1, y.k1) or (x.k2, y.k2) (or whatever the preferred syntax is) instead of: a.sort(lambda x,y : cmp(x.k1, y.k1) or (x.k2, y.k2)) Not that I find either form better than the other, but I do find both better than have to define a named function. I am going to see if I can make a PEP for this. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] code blocks using 'for' loops and generators
Samuele Pedroni wrote:
OTOH a suite-based syntax for thunks can likely not work as a substitute of
lambda for cases like:
f(lambda: ..., ...)
where the function is the first argument, and then there are further
arguments.
I do not see why you say suite-based thunks cannot be used in the case in
which the function takes more than one argument. Using the syntax I
described earlier, they work in just that way:
def pickled_file(thunk, name):
...
new_data = thunk(pickle.load(f))
...
with greetings from pickled_file('greetings.pickle'):
...
value greetings
One can make an analogy with self. Both the thunk and self can be
passed automatically as the first argument, and further arguments can
follow. In this way, functions that already take a thunk as a first
argument (such as sort) can be re-used without modification.
Apart this one very hard problem, it would be nice to be able to define
and pass more thunks simultaneously. In particular a more concise syntax for
def setx(self, v): self._x = v
def getx(self): return self._x
x = property(getx,setx)
was considered in the past discussions about the topic a worthy goal.
Here I can make another analogy with self. Just as python does not give
syntactic support for multiple dispatch because (I assume) it would
require major syntax changes for something that would be rarely used, I do
not think it is worth it to give syntactic support for multiple thunks.
If only a fraction epsilon of functions take a single thunk, then I
would guess that epsilon**2 take two thunks, and it is not worth
creating syntax for such a small number of cases, especially if that
syntax compromises what would otherwise be a much cleaner syntax for the
single thunk case.
-Brian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] code blocks using 'for' loops and generators
On Sun, 13 Mar 2005, Greg Ewing wrote: Brian Sabbey wrote: I prefer re-using the 'for' loop for this purpose because it allows the problem to be solved in a general way by re-using a structure with which most users are already familiar, and uses generators, which are easier to use in this case than defining a class with __exit__, __enter__, etc. But this is an abuse of both the for-loop and the generator. It's using a for-loop for something that does not loop and is never intended to loop, and it's using yield to do something other than producing a value for consumption. I'd rather see a new mechanism altogether for this very different use case. The problem with creating a new mechanism is that sometimes you will want to loop. For example, reading a bunch of items from a shared resource, modifying them, and sending them back. A new, non-looping mechanism will not be adequate for this because it cannot loop, and a 'for' loop will not be adequate for the same reason it is not adequate now: it can't automatically unlock the resource at the end of the loop. I was looking for a single mechanism to handle all of these cases, but I can see why you and most people would not be so keen on the idea of abusing (I prefer expanding the uses of) 'for' loops this way. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] code blocks using 'for' loops and generators
On Mon, 14 Mar 2005, Greg Ewing wrote:
Brian Sabbey wrote:
The problem with creating a new mechanism is that sometimes you will want
to loop. For example, reading a bunch of items from a shared resource,
modifying them, and sending them back. A new, non-looping mechanism will
not be adequate for this because it cannot loop,
If there is a mechanism for passing a code block as
a thunk to an arbitrary function, the function is
free to loop or not as it sees fit. I'd just prefer
the spelling of it didn't force you to make it
look like it's looping when it's not.
I think you're right. How about something like below? In the same way
that self is passed behind the scenes as the first argument, so can
the thunk be. (Many ideas taken from around [1])
def stopwatch(thunk):
t = time.time()
thunk()
return t - time.time()
with stopwatch() result dt:
a()
b()
print 'it took', dt, 'seconds to compute'
=
def pickled_file(thunk, name):
f = open(name)
new_data = thunk(pickle.load(f))
f.close()
f = open(name, 'w')
pickle.dump(new_data, f)
f.close()
with greetings from pickled_file('greetings.pickle'):
greetings.append('hello')
greetings.append('howdy')
value greetings
[1] http://mail.python.org/pipermail/python-dev/2003-February/032732.html
-Brian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] code blocks using 'for' loops and generators
On Fri, 11 Mar 2005, Josiah Carlson wrote:
My first reaction to the proposal was ick. Why? Because every time
I've seen a mechanism for modifying the internals of generators
constructed using yield, the syntax has been awful; in my opinion
(whether my opinion means anything is another matter), the syntax you
propose is quite awful,
I find it quite natural. Stuff on the right of 'yield' is going out,
stuff on the left is coming in. Since 'yield' is different than return in
that it marks a spot at which execution both leaves and re-enters the
frame, it makes sense that 'yield' should have a syntax that indicates as
much.
and the functionality you desire does not
require syntax modification to be possible.
Was the functionality provided by generators or decorators or anything
else impossible before those were introduced? Of course not. The point
is to make things easier and more natural, not to enable some previously
impossible functionality.
Strawman 1: the syntax
def pickled_file(name):
f = open(name, 'r')
l yield pickle.load(f)
^
--
|f.close()
|f = open(name, 'w')
|pickle.dump(l, f)
|f.close()
|
While this is currently a syntax error, it is not clear that an
assignment is happening /at all/, and I don't believe it would be /even
if/ if were documented, and every time someone opened up Python it said
This is an assignment! with an example. It looks too magical to me
(and from a guy who had previously proposed 'Some' as the opposite of
'None', that's saying something).
Perhaps you are right, I don't know. It seems to me that most people
would have to see the syntax once to know exactly what is going on, but I
certainly don't know that for sure. Either way, I'd hate to have all my
suggestions dismissed because of the syntax of this one piece.
Strawman 2: putting data back into a generator
def pickled_file(name):
f = open(name, 'r')
yield pickle.load(f)
f.close()
f = open(name, 'w')
pickle.dump(l, f)
f.close()
Keep your code, except toss that leading 'l'.
for l in pickled_file('greetings.pickle'):
l.append('hello')
l.append('howdy')
Toss that 'continue l', and your code will work (as long as both the
function and the for are sitting in the same namespace).
But they're *not* in the same namespace necessarily. That is entirely the
point. One is changing scope but has no clean way to pass values. How is
making 'l' some (more) global variable possibly a clearer way to pass it
to the generator? Your argument is like saying one does not need to
return values from a function because we could always just use a global
variable to do it.
Hrm, not good enough? Use a Queue, or use another variable in a
namespace accessable to both your function and your loop.
Again, I entirely realize it's possible to do these things now, but that
is not the point.
I'm sorry if this email comes off harsh,
I'm just relieved someone responded :)
-Brian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] code blocks using 'for' loops and generators
On Sat, 12 Mar 2005, Steven Bethard wrote:
The goals behind this seem a lot like the goals of PEP 288[1]. I
remember discussions suggesting code like:
def gen():
a, b, c=3 = yield 1
yield a + b*c
g = gen()
print g.next() # prints 1
print g.next(1, 2) # prints 7
But as you can see, this creates a weird asymmetry because the last
yield throws away its arguments, and depending on how the generator is
written, different calls to next may require a different number of
arguments. This means that, unless the code is extremely well
documented, you have to read the source code for the generator to know
how to call it.
The intention of my proposal was for using generators with 'for' loops.
In this case, the generator runs to completion, so the arguments to the
last yield are never thrown away. If 'next' were not able to take any
arguments, that would be compatible with my proposal.
Also, there was the issue that there is an asymmetry because the first
call to 'next' does not take any arguments. This asymmetry does not
exist, however, when using the generator in a 'for' loop, because there is
no first call to 'continue' in such a case.
Because of these and other complications, I believe the PEP is now
lobbying for a way to get the generator instance object and a way to
cause an exception to be thrown from outside the generator. Take a
look and see if the PEP might meet your needs -- I haven't seen much
action on it recently, but it seems much less invasive than your
proposal...
This PEP solves similar problems, yes. And I would agree that my proposal
is much more invasive on python's implementation. From the users' point
of view, however, I think it is much less invasive. For example, no doubt
there will be many users who write a generator that is to be used in a
'for' loop and are baffled that they receive a syntax error when they try
to write some try/finally cleanup code. With the PEP, they would have to
figure out that they have to use the 'throw' method of generators to
trigger cleanup code (and then have to remember to call it each time they
are done with the generator). With this proposal, try/finally would just
work as they expect and they would be non the wiser.
def pickled_file(name):
self = mygen.get_instance()
f = open(name, 'r')
yield pickle.load(f)
f.close()
f = open(name, 'w')
pickle.dump(self.l, f)
f.close()
And this would be written something like:
gen = pickled_file('greetings.pickle')
for l in gen:
l.append('hello')
l.append('howdy')
gen.l = l
Personally, I find this use of a generator thoroughly confusing, and I
don't see what you gain from it. The PEP 288 examples are perhaps
somewhat more convincing though...
The disadvantage of doing it this way (or with a class wrapping the
generator) is that it is implicit. If I were reading the pickled_file
code, I would have no idea where the self.l comes from. If it is coming
from the 'for' loop, why not just be able to explicitly say that?
I agree that this is a confusing way to use generators. But it is the
expected way to use code blocks as found in other languages. It would
take some getting used to that 'for' can be used this way, but I think it
would be worth it.
-Brian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] code blocks using 'for' loops and generators
On Sat, 12 Mar 2005, Steven Bethard wrote:
Brian Sabbey [EMAIL PROTECTED] wrote:
I agree that this is a confusing way to use generators. But it is the
expected way to use code blocks as found in other languages. It would
take some getting used to that 'for' can be used this way, but I think it
would be worth it.
I guess I need some more convincing... I don't find your approach[*], e.g.
def pickled_file(name):
f = open(name, 'r')
data yield pickle.load(f)
f.close()
f = open(name, 'w')
pickle.dump(data, f)
f.close()
for data in pickled_file('greetings.pickle'):
data.append('hello')
data.append('howdy')
continue data
any clearer than, say:
class PickledFile(object):
def __init__(self, name):
self.name = name
f = open(self.name, 'r')
self.data = pickle.load(f)
f.close()
def close(self):
f = open(self.name, 'w')
pickle.dump(self.data, f)
f.close()
p = PickledFile('greetings.pickle')
p.data.extend(['hello', 'howdy'])
p.close()
Note that I'm not using the iteration construct (for-in) because I'm
not really doing an iteration here. Pehaps I could be taught to read
for-in otherwise, but without an obvious benefit for doing so, I'm
really not inclined to.
In the real world, I need to lock and unlock the pickled files. So if I
forget to call p.close(), then it would be a problem. I would need a
try/finally block. If I could use the above 'for' loop approach, I am
able to encapsulate the try/finally code.
This is the same problem that is addressed in PEP 310. Copied from the
PEP, here is the basic idea:
The syntax of a 'with' statement is as follows::
'with' [ var '=' ] expr ':'
suite
This statement is defined as being equivalent to the following
sequence of statements:
var = expr
if hasattr(var, __enter__):
var.__enter__()
try:
suite
finally:
var.__exit__()
I prefer re-using the 'for' loop for this purpose because it allows the
problem to be solved in a general way by re-using a structure with which
most users are already familiar, and uses generators, which are easier to
use in this case than defining a class with __exit__, __enter__, etc.
-Brian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] code blocks using 'for' loops and generators
On Sat, 12 Mar 2005, Josiah Carlson wrote: I stand by my clever hack statement, and I will agree to disagree with you on it, like I agree to disagree with you about both the necessity of passing arbitrary values back into a generator (I think it is poor style, and too much brain overhead to wonder where data is coming from), and the necessity of a new syntax to make such things easier (if (ab)using generators in such a way makes /anything/ easier). I will also agree to disagree. If I may also summarize, you see such syntax as magical, non-obvious and unnecessary, and I find it clean and useful. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] code blocks using 'for' loops and generators
I would like to get some feedback on a proposal to introduce
smalltalk/ruby-like code blocks to python. Code blocks are, among other
things, a clean way to solve the acquire/release problem [1][2]. This
proposal also touches on some of the problems PEP 288 [3] deals with.
The best discussion I have been able to find on this topic is in the
thread of ref. [4].
Since generators, when used with 'for' loops, are so similar to code
blocks [5], one can imagine two ways to implement code blocks in python:
(1, parallel) keep them as similar as possible to 'for' loops so that a
normal user doesn't distinguish the two, or (2, orthogonal) make them so
much different than 'for' loops that the normal user doesn't notice the
similarities. Anything between these extremes will probably be confusing.
Here I will describe an attempt at (1).
(I) Give generators a __call__ method as an alternative to 'next'.
Method __call__ should take a single parameter: a function. Using
__call__ will cause the generator to start executing normally, but when a
'yield' is reached, the generator will invoke the function passed to
__call__ instead of activating the currently implemented 'yield'
mechanism.
Since __call__ will be an alternative to 'next', it will raise an
exception if 'next' has already been called and vice-versa. Also, any
calls to 'next' will raise an exception if there is a 'yield' in the try
of a try/finally. Such yields will no longer trigger a SyntaxError
because they will not a problem when using __call__.
(II) Have a method of generators, __blockcall__, which will be equal to
__call__, but will only exist if (1) the generator contains a try/finally
try yield, or (2) the user explicitly defines it, for example, with a
function decorator (@completion_required would be a descriptive name).
Have 'for' loops use __blockcall__ if it is available, and __iter__
otherwise. Pass to __blockcall__ the block of code in the 'for' loop.
Scope rules for the passed block of code should mimic current 'for' loop
behavior. Behavior of 'break' and 'return' should be mimicked, perhaps
with special exceptions catchable only by the 'for' loop. Mimicking
'yield' will be a problem unless/until multi-level yields are allowed.
(performance and implementation difficulties for all of this? I don't
know).
The thunk shouldn't be savable for later use because the 'for' loop will
no longer be around to deal with 'break' and 'return'. This means that
__blockcall__ will not be implementable as a function that takes a
function as an argument.
(III) Allow 'continue' to pass values to 'yield' (something similar
previously rejected here [6]). As far as I know, all other control
statements that transfer execution to a different frame (yield, return,
raise) pass values, and I don't see why this case should be any different.
I do not see such a mechanism as gimmicky; being able to cleanly pass
values when changing scope is an inherent part of nearly every programming
language.
As an example of the syntax I am suggesting, here is something I was
desiring recently, a generator to open, unpickle, repickle and close a
file:
def pickled_file(name):
f = open(name, 'r')
l yield pickle.load(f)
f.close()
f = open(name, 'w')
pickle.dump(l, f)
f.close()
The unpickled object is sent to the caller at the yield statement, and the
modified object is received back at the same statement. Note the
suggested 'yield' syntax and the conspicuous absence of '='. This syntax
is backwardly compatible with current yield syntax. Also, this syntax
does not require yield to appear as a function; it is still clear that
this is a unique control-flow statement.
This function would be used like this:
for l in pickled_file('greetings.pickle'):
l.append('hello')
l.append('howdy')
continue l
The above code would have the same effect as:
def named(l):
l.append('hello')
l.append('howdy')
return l
pickled_file('greetings.pickle')(named)
(IV) Allow 'yield' to return no value; in this case a new keyword,
'with', will be required instead of an awkward 'for':
with f(): instead of for in f():
(V) For the same reasons as in (III), allow generators to return values.
These values can be sent with the StopIteration exception if 'next' is
being used for iteration. An obvious syntax for receiving these values is
shown by this example:
with dt = stopwatch():
f()
g()
print 'it took', dt, 'seconds'
Although with stopwatch() result dt: might not be so bad.
[1] PEP 310 Reliable Acquisition/Release Pairs
http://www.python.org/peps/pep-0310.html
[2] PEP 325 Resource-Release Support for Generators
http://www.python.org/peps/pep-0325.html
[3] PEP 288 Generators Attributes and Exceptions
http://www.python.org/peps/pep-0288.html
[4] http://mail.python.org/pipermail/python-dev/2003-February/032800.html
[5]
