John wrote:
Hi,
I just read a few pages of tutorial on list comprehenion and generator
expression. From what I gather the difference is "[ ]" and "( )" at the
ends, better memory usage and the something the tutorial labeled as "lazy
evaluation". So a generator 'yields'. But what is it yielding too?
John
A list comprehension builds a whole list at one time. So if the list
needed is large enough in size, it'll never finish, and besides, you'll
run out of memory and crash. A generator expression builds a function
instead which *acts* like a list, but actually doesn't build the values
till you ask for them. But you can still do things like
for item in fakelist:
and it does what you'd expect.
You can write a generator yourself, and better understand what it's
about. Suppose you were trying to build a "list" of the squares of the
integers between 3 and 15. For a list of that size, you could just use
a list comprehension. But pretend it was much larger, and you couldn't
spare the memory or the time.
So let's write a generator function by hand, deliberately the hard way.
def mygen():
i = 3
while i < 16:
yield i*i
i += 1
return
This function is a generator, by virtue of that yield statement in it.
When it's called, it does some extra magic to make it easy to construct
a loop.
If you now use
for item in mygen():
print item
Each time through the loop, it executes one more iteration of the
mygen() function, up to the yield statement. And the value that's put
into item comes from the yield statement.
When the mygen() function returns (or falls off the end), it actually
generates a special exception that quietly terminates the for/loop.
Now, when we're doing simple expressions for a small number of values,
we should use a list comprehension. When it gets big enough, switch to
a generator expression. And if it gets complicated enough, switch to a
generator function. The point here is that the user of the for/loop
doesn't care which way it was done.
Sometimes you really need a list. For example, you can't generally back
up in a generator, or randomly access the [i] item. But a generator is
a very valuable mechanism to understand.
For a complex example, consider searching a hard disk for a particular
file. Building a complete list might take a long time, and use a lot of
memory. But if you use a generator inside a for loop, you can terminate
(break) when you meet some condition, and the remainder of the files
never had to be visited. See os.walk()
DaveA
_______________________________________________
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
