On Mon, May 3, 2021 at 10:08 PM Steven D'Aprano <st...@pearwood.info> wrote:
>
> On Mon, May 03, 2021 at 09:04:51PM +1000, Chris Angelico wrote:
>
> > > My understanding of the situation is that the list comprehension [
> > > x*x for x in range(5) ] is a shorthand for list( x*x for x in
> > > range(5) ).
> >
> > Sorta-kinda. It's not a shorthand in the sense that you can't simply
> > replace one with the other,
>
> Only because the `list` name could be shadowed or rebound to something
> else. Syntactically and functionally, aside from the lazy vs eager
> difference, a comprehension is a comprehension and there is nothing
> generator comprehensions can do that list comprehensions can't.
I mention the rebinding, but I'm not ruling out the possibility of
other distinctions, perhaps due to order of execution.
> > but they do have very similar behaviour,
> > yes. A genexp is far more flexible than a list comp,
>
> Aside from the lazy nature of generator comprehensions, what else?
Yielding is bidirectional. You won't see it when you just pass it to
the list() constructor, but the genexp can have values sent back into
it. That entails some extra machinery that is completely unnecessary
for building a list, although, as I mentioned...
> > so the compiled
> > bytecode for list(genexp) has to go to a lot of unnecessary work to
> > permit that flexibility, whereas the list comp can simplify things
> > down.
... it's mainly just a matter of simplifications.
> I don't think so. The bytecode in 3.9 is remarkably similar.
Yes, it looks similar.
> >>> dis.dis('list(spam for spam in eggs)')
> Disassembly of <code object <genexpr> at 0x7fc185ce0870, file "<dis>",
> line 1>:
> 1 0 LOAD_FAST 0 (.0)
> >> 2 FOR_ITER 10 (to 14)
> 4 STORE_FAST 1 (spam)
> 6 LOAD_FAST 1 (spam)
> 8 YIELD_VALUE
> 10 POP_TOP
YIELD_VALUE followed by POP_TOP is your clue that it's bidirectional.
The comprehension simply appends onto the list immediately. The genexp
has to have two completely separate scopes and switch between them;
the list comp runs everything in the same inner scope, building up the
list.
> The bytecode for the list comp `[spam for spam in eggs]` is only three
> bytecodes shorter, so that doesn't support your comment about "a lot of
> unnecessary work".
"Three bytecodes shorter" conceals the fact that some bytecodes do a
LOT of work. Look into how much work it takes to restart a generator,
and compare that to the bytecode "APPEND_LIST".
> As far as runtime efficiency, list comps are a little faster. Iterating
> over a 1000-item sequence is 33% faster for a list comp, but for a
> 100000-item sequence that drops to 25% faster. But as soon as you do a
> significant amount of work inside the comprehension, that work is likely
> to dominate the other costs.
How often are you doing a significant amount of work inside a comprehension?
> There's definitely some overhead needed to support starting and stopping
> a generator, but we can argue that is an implementation detail. A
> sufficiently clever interpreter could avoid that overhead.
No, it can't - except by rewriting it as a list comp, and I'm not
certain that there wouldn't be timing distinctions. A genexp cannot
skip the overhead of being a generator.
> > That said, I think the only way you'd actually detect a
> > behavioural difference is if the name "list" has been rebound.
>
> That and timing.
Yes, I don't count that as a behavioural difference. Nor memory usage,
within reason.
ChrisA
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/YTITKM43RLWYPXDHWMZ6T4CBJH44BUZY/
Code of Conduct: http://python.org/psf/codeofconduct/
