Patches item #1548388, was opened at 2006-08-29 03:33
Message generated for change (Comment added) made by rhettinger
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1548388&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Core (C code)
Group: Python 3000
Status: Open
Resolution: None
Priority: 5
Submitted By: Georg Brandl (gbrandl)
Assigned to: Georg Brandl (gbrandl)
Summary: set comprehensions
Initial Comment:
This is a big one:
* cleanup grammar; unifies listcomp/genexp grammar
which means that [x for x in 1, 2] is no longer valid
* cleanup comprehension compiling code (unifies all AST
code for the three comprehensions and most of the
compile.c code)
* add set comprehensions
This patch modifies list comprehensions to be
implemented more like generator expressions: in a
separate function, which means that the loop variables
will not leak any more.
----------------------------------------------------------------------
>Comment By: Raymond Hettinger (rhettinger)
Date: 2006-09-08 12:13
Message:
Logged In: YES
user_id=80475
Genexps necessarily need a separate stack frame to achieve
saved execution state (including the instruction pointer and
local variable). Also, it was simplest to implement genexps
in terms of the existing and proven code for regular generators.
For list and set comps, I think you can take a simpler
approach and just rename the inner loop variable to
something invisible. That will make it faster, make the
disassemby readable, and make it easier to follow in pdb.
Also, we get to capitalize on proven code -- they only
difference is that the induction variable won't be visible
to surrounding code.
Since what you have works, I would say just check it in;
however, it would probably never get touched again and an
early, arbitrary design choice would get set in stone. My
bet is that the renaming approach will result in a much
simpler patch.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2006-09-06 10:36
Message:
Logged In: YES
user_id=6380
I always assumed that the genexps *require* being a function
because that's the only way to create a generator. But that
argument doesn't apply to listcomps.
That's about all I know of the implementation of these.. :-(
Have you asked python-dev?
----------------------------------------------------------------------
Comment By: Georg Brandl (gbrandl)
Date: 2006-09-06 02:03
Message:
Logged In: YES
user_id=849994
It is complete, it works and it does not leak the loop
variable(s).
The question is whether it is okay for listcomps and
setcomps to be in their own anonymous function, which slows
listcomps down compared to the 2.x branch.
I don't know why the function approach was taken for
genexps, but I suspect it was because who implemented it
then saw this as the best way to hide the loop variable.
Perhaps somebody else more familiar with the internals and
the previous discussions can look over it.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2006-09-06 01:48
Message:
Logged In: YES
user_id=6380
Do you think this is ready to be checked in, or are you
still working on it?
----------------------------------------------------------------------
Comment By: Georg Brandl (gbrandl)
Date: 2006-09-01 04:38
Message:
Logged In: YES
user_id=849994
Since you can put anything usable as an assignment target
after the "for" of a listcomp, just renaming might be
complicated.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2006-08-31 18:40
Message:
Logged In: YES
user_id=6380
+1.
Would this cause problems for abominations like this though?
>>> a=[1]
>>> list(tuple(a) for a[0] in "abc")
[('a',), ('b',), ('c',)]
>>> a
['c']
>>>
----------------------------------------------------------------------
Comment By: Raymond Hettinger (rhettinger)
Date: 2006-08-31 18:15
Message:
Logged In: YES
user_id=80475
Would it be an oversimplfication for list and set comps to
keep everything in one code block and just hide the list
loop variables by renaming them: x --> __[x]
That approach would only require a minimal patch, and it
would make for a cleaner disassembly.
----------------------------------------------------------------------
Comment By: Georg Brandl (gbrandl)
Date: 2006-08-31 14:55
Message:
Logged In: YES
user_id=849994
Attaching slightly revised patch and bytecode comparison.
----------------------------------------------------------------------
Comment By: Raymond Hettinger (rhettinger)
Date: 2006-08-29 17:30
Message:
Logged In: YES
user_id=80475
Can you post a before and disassembly of some list and set
comprehensions.
----------------------------------------------------------------------
Comment By: Georg Brandl (gbrandl)
Date: 2006-08-29 14:09
Message:
Logged In: YES
user_id=849994
test_compiler and test_transformer fail because the compiler
package hasn't been updated yet.
test_dis fails because list comprehensions now generate
different bytecode.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2006-08-29 12:59
Message:
Logged In: YES
user_id=6380
Nice!
I see failures in 4 tests:
test_compiler test_dis test_transformer test_univnewlines
test_univnewlines is trivial (it's deleting a variable
leaked out of a list comprehension); haven't looked at the
rest in detail
----------------------------------------------------------------------
Comment By: Georg Brandl (gbrandl)
Date: 2006-08-29 03:34
Message:
Logged In: YES
user_id=849994
The previously attached patch contains only the important
files. The FULL patch (attached now) also contains syntax
fixes in python files so that the test suite is mostly passing.
Note that the compiler package isn't ready yet.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1548388&group_id=5470
_______________________________________________
Patches mailing list
Patches@python.org
http://mail.python.org/mailman/listinfo/patches
