On 6/26/2018 10:36 PM, Guido van Rossum wrote:
[This is my one response today]
Thank you for clearly presenting how you see 'comprehension', 'generator
expression' and by implication 'equivalent code'. The latter can either
be a definition or an explanation. The difference is subtle but real,
and, I believe, the essence of the disagreement over iteration
variables. If the code equivalent to a comprehension is its definition,
like a macro expansion, then survival of the iteration variable is to be
expected. If the equivalent code is an explanation of the *result* of
evaluating a *self-contained expression*, then leakage is easily seen a
wart, just as leakage of temporaries from any other expression would be.
My interpretation of what you say below is that you always wanted, for
instance, [i*i for i in iterable] == [j*j for j in iterable] to be true,
and saw the leakage making this not quite true as a wart. In other
words, within comprehensions (including generator expressions)
iterations names should be regarded as dummy placeholders and not part
of the value.
If this is correct, the list comprehension syntax could have been
[\0 * \0 for \0 in iterable]
with \1, \2, ... used as needed. (I am using the regex back-reference
notation in a way similar to the use of str.format forward reference
notation.)
I will stop here for now, as it is 1:30 am for me.
Terry
On Mon, Jun 25, 2018 at 12:40 PM Terry Reedy <tjre...@udel.edu
<mailto:tjre...@udel.edu>> wrote:
On 6/24/2018 7:25 PM, Guido van Rossum wrote:
> I'd wager that the people who might be most horrified about it
the (b) scoping rule change
> would be people who feel strongly that the change to the
> comprehension scope rules in Python 3 is a big improvement,
I might not be one of those 'most horrified' by (b), but I increasingly
don't like it, and I was at best -0 on the comprehension scope change.
To me, iteration variable assignment in the current scope is a
non-problem. So to me the change was mostly useless churn. Little
benefit, little harm. And not worth fighting when others saw a benefit.
Fair enough, and by itself this might not have been enough reason to
make the change. But see below.
However, having made the change to nested scopes, I think we should
stick with them. Or repeal them. (I believe there is another way to
isolate iteration names -- see below). To me, (b) amounts to half
repealing the nested scope change, making comprehensions half-fowl,
half-fish chimeras.
That depends on how you see it -- to me (b) just means that there's an
implicit nonlocal[1] to make the assignment have the (desirable)
side-effect.
The key thing to consider here is whether that side-effect is in fact
desirable. For me, the side-effect of the comprehension's loop control
variable was never desirable -- it was just an implementation detail
leaking out. (And that's different from leaking a regular for-loop's
control variable -- since we have 'break' (and 'else') there are some
legitimate use cases. But comprehensions try to be expressions, and here
the side effect is at best useless and at worst a nasty surprise.)
> and who are familiar with the difference in implementation
> of comprehensions (though not generator expressions) in Python 2
vs. 3.
That I pretty much am, I think. In Python 2, comprehensions (the fish)
were, at least in effect, expanded in-line to a normal for loop.
Generator expressions (the fowls) were different. They were, and still
are, expanded into a temporary generator function whose return value is
dropped back into the original namespace. Python 3 turned
comprehensions (with 2 news varieties thereof) into fowls also,
temporary functions whose return value is dropped back in the original
namespace. The result is that a list comprehension is equivalent to
list(generator_ expression), even though, for efficiency, it is not
implemented that way. (To me, this unification is more a benefit than
name hiding.)
Right, and this consistency convinced me that the change was worth it. I
just really like to be able to say "[... for ...]" is equivalent to
"list(... for ...)", and similar for set and dict.
(b) proposes to add extra hidden code in and around the temporary
function to partly undo the isolation.
But it just adds a nonlocal declaration. There's always some hidden code
('def' and 'return' at the very least).
list comprehensions would no
longer be equivalent to list(generator_expression), unless
generator_expressions got the same treatment, in which case they would
no longer be equivalent to calling the obvious generator function.
Breaking either equivalence might break someone's code.
Ah, there's the rub! I should probably apologize for not clarifying my
terminology more. In the context of PEP 572, when I say "comprehensions"
I include generators! PEP 572 states this explicitly
(https://github.com/python/peps/blame/master/pep-0572.rst#L201-L202).
Certainly PEP 572 intends to add that implicit nonlocal to both
comprehensions and generator expressions. (I just got really tired of
writing that phrase over and over, and at some point I forgot that this
is only a parenthetical remark added in the PEP's latest revision, and
not conventional terminology -- alas. :-)
Part (b) of PEP 572 does several things of things to *retain* consistency:
- The target of := lives in the same scope regardless of whether it
occurs in a comprehension, a generator expression, or just in some other
expression.
- When it occurs in a comprehension or generator expression, the scope
is the same regardless of whether it occurs in the "outermost iterable"
or not.
If we didn't have (b) the target would live in the comprehension/genexpr
scope if it occurred in a comprehension/genexp but outside its
"outermost iterable", and in the surrounding scope otherwise.
---
How loop variables might be isolated without a nested scope: After a
comprehension is parsed, so that names become strings, rename the loop
variables to something otherwise illegal. For instance, i could become
'<i>', just as lambda becomes '<lambda>' as the name of the resulting
function. Expand the comprehension as in Python 2, except for deleting
the loop names along with the temporary result name.
Assignment expressions within a comprehension would become assignment
expressions within the for loop expansion and would automatically
add or
replace values in the namespace containing the comprehension. In other
words, I am suggesting that if we want name expressions in
comprehensions to act as they would in Python 2, then we should
consider
reverting to an altered version of the Python 2 expansion.
Possibly this is based on a misunderstanding of my use of
"comprehensions". Also, since your trick can only be used for
list/set/dict comprehensions, but not for generator expressions (at
least I assume you don't want it there) it would actually *reduce*
consistency between list/set/dict comprehensions and generator expressions.
---
In any case, I think (b) should be a separate PEP linked to a PEP for
(a). The decision for (a) could be reject (making (b) moot), accept
with (b), or accept unconditionally (but still consider (b)).
For me personally, (b) makes the PEP more consistent, so I'm not in
favor of breaking up the PEP. But we can certainly break up the
discussion -- that's why I started using the labels (a) and (b).
----------
[1] Sometimes it's an implicit global instead of an implicit nonlocal --
when there's already a global for the same variable in the target scope.
--
--Guido van Rossum (python.org/~guido <http://python.org/~guido>)
--
Terry Jan Reedy
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
