Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Wed, Jun 27, 2018 at 12:01 PM, Eric V. Smith  wrote:

> >>> def test():
> >>>spam = 1
> >>>ham = 2
> >>>vars = [key1+key2 for key1 in locals() for key2 in locals()]
> >>>return vars
> >>>
> >>> Wanna guess what that's gonna return?
>

Guessing aside, messing around with locals() isn't really helpful for the
usual case of common code.

But the real problem I see with all of this the distinction between
generator expressions and comprehensions:

"""
an assignment expression occurring in a list, set or dict comprehension or
in a generator expression (below collectively referred to as
"comprehensions") binds the target in the containing scope, honoring a
nonlocal or global declaration for the target in that scope, if one exists.
For the purpose of this rule the containing scope of a nested comprehension
is the scope that contains the outermost comprehension. A lambda counts as
a containing scope.
"""

It seems everyone agrees that scoping rules should be the same for
generator expressions and comprehensions, which is a good reason for
python3's non-leaking comprehensions:

(py2)

In [*5*]: i = 0


In [*6*]: l = [i *for* i *in* range(3)]


In [*7*]: i

Out[*7*]: 2


In [*8*]: i = 0


In [*9*]: g = (i *for* i *in* range(3))


In [*10*]: i

Out[*10*]: 0


In [*11*]: *for* j *in* g:

...: *pass*

...:


In [*12*]: i

Out[*12*]: 0

so comprehensions and generator expressions behave differently -- not great.

(py3)


In [*4*]: i = 0


In [*5*]: l = [i *for* i *in* range(3)]


In [*6*]: i

Out[*6*]: 0


In [*7*]: g = (i *for* i *in* range(3))


In [*8*]: i

Out[*8*]: 0


In [*9*]: list(g)

Out[*9*]: [0, 1, 2]


In [*10*]: i

Out[*10*]: 0

The loop name doesn't "leak" and comprehensions and generator expressions
are the same this regard -- nice.

So what about:

l = [x:=i for i in range(3)]

vs

g = (x:=i for i in range(3))

Is there any way to keep these consistent if the "x" is in the regular
local scope?

Note that this thread is titled "Informal educator feedback on PEP 572".

As an educator -- this is looking harder an harder to explain to newbies...

Though easier if any assignments made in a "comprehension" don't "leak out".

Which does not mean that we'd need a "proper" new local scope (i.e.
locals() returning something new) -- as long as the common usage was
"intuitive".

>> I'm not singling out Chris here, but these discussions would be easier
> >> to follow and more illuminating if the answers to such puzzles were
> >> presented when they're posed.
>

well, I think the point there was that it wasn't obvious without running
the code -- and that point is made regardless of the answer.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Guido]
> ..
> Given that definition of `__parentlocal`, in first approximation the
> scoping rule proposed by PEP 572 would then be: In comprehensions
> (which in my use in the PEP 572 discussion includes generator
> expressions) the targets of inline assignments are automatically
> endowed with a `__parentlocal` declaration, except inside the
> "outermost iterable" (since that already runs in the parent scope).

If this has to be done ;-) , I suggest removing that last exception.  That
is, "[all] targets of inline assignments in comprehensions are declared
__parentlocal", period, should work fine for (b).  In case one appears in
the outermost iterable of the outermost comprehension, I believe such
declaration is merely semantically redundant, not harmful.

Where "redundant" means someone is so familiar with the implementation that
the scope implications of "already runs in the parent scope" are
immediately clear.  For someone muddy about that, it would be a positive
help to have the intent clarified  by removing the exception.

Plus 99% of the point of "parentlocal" seemed to be to allow mindless
("uniform") by-hand translation of nested comprehensions to nested Python
functions, and an exception for the outermost iterable would work against
that intent.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Angelico]

On Thu, Jun 28, 2018 at 11:29 AM, Ivan Pozdeev via Python-Dev
 wrote:
> On 28.06.2018 2:44, Greg Ewing wrote:
>>
>> Ivan Pozdeev via Python-Dev wrote:
>>>
>>> for me, the primary use case for an assignment expression is to be able
>>> to "catch" a value into a variable in places where I can't put an assignment
>>> statement in, like the infamous `if re.match() is not None'.
>>
>>
>> This seems to be one of only about two uses for assignment
>> expressions that gets regularly brought up. The other is
>> the loop-and-a-half, which is already adequately addressed
>> by iterators.
>>
>> So maybe instead of introducing an out-of-control sledgehammer
>> in the form of ":=", we could think about addressing this
>> particular case.
>>
>> Like maybe adding an "as" clause to if-statements:
>>
>>if pattern.match(s) as m:
>>   do_something_with(m)
>>
>
> I've skimmed for the origins of "as" (which I remember seeing maybe even
> before Py3 was a thing) and found this excellent analysis of modern
> languages which is too a part of the PEP 572 discussion:
> https://mail.python.org/pipermail/python-ideas/2018-May/050920.html
>
> It basically concludes that most recently-created languages do not have
> assignment expressions; they rather allow assignment statement(s?) before
> the tested expression in block statements (only if/while is mentioned. `for'
> is not applicable because its exit condition in Python is always the
> iterable's exhaustion, there's nothing in it that could be used as a
> variable).
>

Now read this response.

https://mail.python.org/pipermail/python-ideas/2018-May/050938.html

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 28.06.2018 2:44, Greg Ewing wrote:

Ivan Pozdeev via Python-Dev wrote:
for me, the primary use case for an assignment expression is to be 
able to "catch" a value into a variable in places where I can't put 
an assignment statement in, like the infamous `if re.match() is not 
None'.


This seems to be one of only about two uses for assignment
expressions that gets regularly brought up. The other is
the loop-and-a-half, which is already adequately addressed
by iterators.

So maybe instead of introducing an out-of-control sledgehammer
in the form of ":=", we could think about addressing this
particular case.

Like maybe adding an "as" clause to if-statements:

   if pattern.match(s) as m:
  do_something_with(m)



I've skimmed for the origins of "as" (which I remember seeing maybe even 
before Py3 was a thing) and found this excellent analysis of modern 
languages which is too a part of the PEP 572 discussion:

https://mail.python.org/pipermail/python-ideas/2018-May/050920.html

It basically concludes that most recently-created languages do not have 
assignment expressions; they rather allow assignment statement(s?) 
before the tested expression in block statements (only if/while is 
mentioned. `for' is not applicable because its exit condition in Python 
is always the iterable's exhaustion, there's nothing in it that could be 
used as a variable).


It, however, doesn't say anything about constructs that are not block 
statements but are equivalent to them, like the ternary operator. (In 
comprehensions, filter conditions are the bits equivalent to if/while 
statements.)


--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python 3.7.0 is now available! (and so is 3.6.6)

[Ned Deily]

On behalf of the Python development community and the Python 3.7 release
team, we are pleased to announce the availability of Python 3.7.0.
Python 3.7.0 is the newest feature release of the Python language, and
it contains many new features and optimizations. You can find Python
3.7.0 here:

https://www.python.org/downloads/release/python-370/

Most third-party distributors of Python should be making 3.7.0 packages
available soon.

See the "What’s New In Python 3.7" document
(https://docs.python.org/3.7/whatsnew/3.7.html) for more information
about features included in the 3.7 series. Detailed information about
the changes made in 3.7.0 can be found in its change log. Maintenance
releases for the 3.7 series will follow at regular intervals starting in
July of 2018.

We hope you enjoy Python 3.7!

P.S. We are also happy to announce the availability of Python 3.6.6, the
next maintenance release of Python 3.6:

https://www.python.org/downloads/release/python-366/

Thanks to all of the many volunteers who help make Python Development
and these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the
Python Software Foundation.

https://www.python.org/psf/

--
  Ned Deily
  n...@python.org -- []

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Tim]
 > If the parent has a matching parentlocal declaration for the same
> name then the original

> > really refers to the grandparent - and so on.
>

[Greg]

> Ah, I missed that part, sorry -- I withdraw that particular
> objecttion.
>

Good!  I have another reply that crossed in the mail.



> Still, this seems like a major addition (seeing as it comes
> with a new keyword) whose justification is very little more
> than "it makes explaining comprehension scopes easier".
>
> I agree - it has no other sane use case I can see, and "parentlocal" isn't
_needed_ to capture the intended semantics in by-hand translations of
comprehensions.

I don't even think it makes "explaining" easier.  It doesn't eliminate any
corner cases, it just pushes them into the definition of what
"parentllocal" means.

What it would do is make writing synthetic functions "by hand" to implement
comprehensions more uniform, because "parentlocal" would handle the corner
cases by itself instead of making the programmer figure out when and where
they need to type "nonlocal", "global", and/or cruft to establish a name as
local to a block in which the name otherwise does't appear as a binding
target.

But to the extent that doing such translations by hand is meant to be
"educational", it's more educational to learn how to do that stuff yourself.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Ivan Pozdeev via Python-Dev wrote:
for me, the primary use case for an assignment expression 
is to be able to "catch" a value into a variable in places where I can't 
put an assignment statement in, like the infamous `if re.match() is not 
None'.


This seems to be one of only about two uses for assignment
expressions that gets regularly brought up. The other is
the loop-and-a-half, which is already adequately addressed
by iterators.

So maybe instead of introducing an out-of-control sledgehammer
in the form of ":=", we could think about addressing this
particular case.

Like maybe adding an "as" clause to if-statements:

   if pattern.match(s) as m:
  do_something_with(m)

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

> [Nick Coghlan]

> I'm OK with a target scope declaration construct having
> > lexical-scope-dependent behaviour - exactly what "nonlocal NAME" will
> > do depends on both the nature of the current scope,



> [Greg Ewing]
>
Yes, but my point is that having an explicit "parentlocal" scope
> declaration doesn't help to make anything more orthogonal,
> because there's no way it can have *exactly* the same effect
> as a comprehension's implicit parent-local scoping.
>

Sure it can - but I already explained that.

This is the analogy to "nonlocal" Nick is making:  neither "nonlocal" nor
"parentlocal" tell you which scope a declared name _does_ belong to.
Instead they both say "it's not this scope" and specify algorithms you can
follow to determine the scope to which the name does belong.

"parentlocal" isn't an accurate name because the owning scope may not be
the parent block at all, and it may even be a synonym for "global".  I
think "by hand" translations of nested comprehensions into nested functions
are clearer _without_ the "parentlocal" invention.- then you have to be
explicit about what the context requires.  Nick hates that because it isn't
uniform.  I like that because I don't want to pretend a non-uniform thing
is uniform ;-)  The only real use case here is synthesizing nested
functions to implement comprehensions/genexps.


In other words, taking a comprehension and manually expanding
> it into a function with parentlocal declarations wouldn't
> give you something exactly equivalent to the original.
> If that's the purpose of having an explicit parentlocal,
> then it fails at that purpose.
>

You can add (a sufficient number of) parentlocal declarations to get the
precise intended semantics.  Then again, that can also be done today
(without the "parentlocal" invention).


>
> If that's *not* the purpose, then I'm not really sure what
> the purpose is, because I can't think of a situation where
> I'd choose to use parentlocal instead of nonlocal with an
> explicit assignment in the outer scope.
>

For example, if the name is declared "global" in the outer scope, you'll
get a compile-time error if you try to declare it "nonlocal" in the
contained scope.  "parentlocal" adjusts its meaning accordingly, becoming a
synonym for "global" in that specific case.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Tim Peters wrote:
If the parent has a 
matching parentlocal declaration for the same name then the original 
really refers to the grandparent - and so on.


Ah, I missed that part, sorry -- I withdraw that particular
objecttion.

Still, this seems like a major addition (seeing as it comes
with a new keyword) whose justification is very little more
than "it makes explaining comprehension scopes easier".

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 28.06.2018 2:31, Greg Ewing wrote:

Steven D'Aprano wrote:
The *very first* motivating example for this proposal came from a 
comprehension.


I think it is both unfortunate and inevitable that the discussion bogged
down in comprehension-hell.


I think the unfortunateness started when we crossed over from
talking about binding a temporary name for use *within* a
comprehension or expression, to binding a name for use *outside*
the comprehension or expression where it's bound.

I've shown in <05f368c2-3cd2-d7e0-9f91-27afb40d5...@mail.mipt.ru> (27 
Jun 2018 17:07:24 +0300) that assignment expressions are fine in most 
use cases without any changes to scoping whatsoever.


So, as Guido suggested in 
 (26 
Jun 2018 19:36:14 -0700), the scoping matter can be split into a 
separate PEP and discussion.



As long as it's for internal use, whether it's in a comprehension
or not isn't an issue.

Tim Peters has also given a couple of good examples of mathematical 
code that would benefit strongly from this feature.


Going back a few months now, they were the examples that tipped me over


Well, I remain profoundly unconvinced that writing comprehensions
with side effects is ever a good idea, and Tim's examples did
nothing to change that.



--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Ivan Pozdeev via Python-Dev wrote:
This isn't as messy as you make it sound if you remember that the 
outermost iterable is evaluated only once at the start and all the 
others -- each iteration.

Anyone using comprehensions has to know this fact.


That fact alone doesn't imply anthing about the *scopes* in which
those iterators are evaluated, however.

Currently the only situation where the scoping makes a difference
is a generator expression that isn't immediately used, and you can
get a long way into your Python career without ever encountering
that case.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 28.06.2018 2:45, Ivan Pozdeev via Python-Dev wrote:

On 28.06.2018 2:31, Ivan Pozdeev via Python-Dev wrote:

On 28.06.2018 1:42, Steven D'Aprano wrote:
On Wed, Jun 27, 2018 at 05:52:16PM +0300, Ivan Pozdeev via 
Python-Dev wrote:



What this means in practice is that assignments will go to different
scopes depending on *where* they are in the comprehension:

 [ expr   for x in iter1  for y in iter2  if cond ...]
 [ BB for x in AA for y in BB if BB ...]

Assignments in the section marked "AA" will be in the local 
scope;
assignments in the BB sections will be in the sublocal scope. 
That's

not too bad, up to the point you try to assign to the same name in
AA and BB. And then you are likely to get confusing hard to
debug UnboundLocalErrors.

This isn't as messy as you make it sound if you remember that the
outermost iterable is evaluated only once at the start and all the
others -- each iteration.
The question isn't *how often* they are evaluated, or how many loops 
you

have, but *what scope* they are evaluated in. Even in a single loop
comprehension, parts of it are evaluated in the local scope and parts
are evaluated in an implicit sublocal scope.


All expressions inside the comprehension other than the initial 
iterable have access to the loop variables generated by the previous 
parts. So they are necessarily evaluated in the internal scope for 
that to be possible.


Since this is too an essential semantics that one has to know to use 
the construct sensibly, I kinda assumed you could make that 
connection...

E.g.:

[(x*y) for x in range(5) if x%2 for y in range(x,5) if not (x+y)%2]
   A  B  C  D   E

C and D have access to the current x; E and A to both x and y.

This means btw that users cannot rely on there being a single internal 
scope, or a scope at all.
The public guarantee is only the access to the loop variables (and, 
with the PEP, additional variables from assignments), of the current 
iteration, generated by the previous parts.


The expressions in the comprehension just somehow automagically 
determine which of the variables are internal and which are local. How 
they do that is an implementation detail.
And the PEP doesn't need to (and probably shouldn't) make guarantees 
here other than where the variables from expressions are promised to be 
accessible.




The overlap between the two is the trap, if you try to assign to the
same variable in the loop header and then update it in the loop body.

Not to mention the inconsistency that some assignments are accessible
from the surrounding code:

 [expr for a in (x := func(), ...) ]
 print(x)  # works

while the most useful ones, those in the body, will be locked up in an
implicit sublocal scope where they are unreachable from outside of the
comprehension:

 [x := something ...  for a in sequence ]
 print(x)  # fails








--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Nick Coghlan wrote:

I'm OK with a target scope declaration construct having
lexical-scope-dependent behaviour - exactly what "nonlocal NAME" will
do depends on both the nature of the current scope,


Yes, but my point is that having an explicit "parentlocal" scope
declaration doesn't help to make anything more orthogonal,
because there's no way it can have *exactly* the same effect
as a comprehension's implicit parent-local scoping.

In other words, taking a comprehension and manually expanding
it into a function with parentlocal declarations wouldn't
give you something exactly equivalent to the original.
If that's the purpose of having an explicit parentlocal,
then it fails at that purpose.

If that's *not* the purpose, then I'm not really sure what
the purpose is, because I can't think of a situation where
I'd choose to use parentlocal instead of nonlocal with an
explicit assignment in the outer scope. Except maybe for the
class-scope situation, which seems like an extremely obscure
reason to introduce a whole new scoping concept with its
own keyword.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 28.06.2018 2:31, Ivan Pozdeev via Python-Dev wrote:

On 28.06.2018 1:42, Steven D'Aprano wrote:
On Wed, Jun 27, 2018 at 05:52:16PM +0300, Ivan Pozdeev via Python-Dev 
wrote:



What this means in practice is that assignments will go to different
scopes depending on *where* they are in the comprehension:

 [ expr   for x in iter1  for y in iter2  if cond   ...]
 [ BB for x in AA for y in BB if BB ...]

Assignments in the section marked "AA" will be in the local scope;
assignments in the BB sections will be in the sublocal scope. 
That's

not too bad, up to the point you try to assign to the same name in
AA and BB. And then you are likely to get confusing hard to
debug UnboundLocalErrors.

This isn't as messy as you make it sound if you remember that the
outermost iterable is evaluated only once at the start and all the
others -- each iteration.

The question isn't *how often* they are evaluated, or how many loops you
have, but *what scope* they are evaluated in. Even in a single loop
comprehension, parts of it are evaluated in the local scope and parts
are evaluated in an implicit sublocal scope.


All expressions inside the comprehension other than the initial 
iterable have access to the loop variables generated by the previous 
parts. So they are necessarily evaluated in the internal scope for 
that to be possible.


Since this is too an essential semantics that one has to know to use 
the construct sensibly, I kinda assumed you could make that connection...

E.g.:

[(x*y) for x in range(5) if x%2 for y in range(x,5) if not (x+y)%2]
   A  B  C  D   E

C and D have access to the current x; E and A to both x and y.

This means btw that users cannot rely on there being a single internal 
scope, or a scope at all.
The public guarantee is only the access to the loop variables (and, with 
the PEP, additional variables from assignments), of the current 
iteration, generated by the previous parts.




The overlap between the two is the trap, if you try to assign to the
same variable in the loop header and then update it in the loop body.

Not to mention the inconsistency that some assignments are accessible
from the surrounding code:

 [expr for a in (x := func(), ...) ]
 print(x)  # works

while the most useful ones, those in the body, will be locked up in an
implicit sublocal scope where they are unreachable from outside of the
comprehension:

 [x := something ...  for a in sequence ]
 print(x)  # fails






--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Intent to accept PEP 561 -- Distributing and Packaging Type Information

[Ethan Furman]

On 06/27/2018 04:11 PM, Guido van Rossum wrote:


Well, with that, I am hereby accepting PEP 561.


Congratulations, Ethan!  :)

--
~Ethan~

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 28.06.2018 1:42, Steven D'Aprano wrote:

On Wed, Jun 27, 2018 at 05:52:16PM +0300, Ivan Pozdeev via Python-Dev wrote:


What this means in practice is that assignments will go to different
scopes depending on *where* they are in the comprehension:

 [ expr   for x in iter1  for y in iter2  if cond   ...]
 [ BB for x in AA for y in BB if BB ...]

Assignments in the section marked "AA" will be in the local scope;
assignments in the BB sections will be in the sublocal scope. That's
not too bad, up to the point you try to assign to the same name in
AA and BB. And then you are likely to get confusing hard to
debug UnboundLocalErrors.

This isn't as messy as you make it sound if you remember that the
outermost iterable is evaluated only once at the start and all the
others -- each iteration.

The question isn't *how often* they are evaluated, or how many loops you
have, but *what scope* they are evaluated in. Even in a single loop
comprehension, parts of it are evaluated in the local scope and parts
are evaluated in an implicit sublocal scope.


All expressions inside the comprehension other than the initial iterable 
have access to the loop variables generated by the previous parts. So 
they are necessarily evaluated in the internal scope for that to be 
possible.


Since this is too an essential semantics that one has to know to use the 
construct sensibly, I kinda assumed you could make that connection...

E.g.:

[(x*y) for x in range(5) if x%2 for y in range(x,5) if not (x+y)%2]
   A  B  C  D   E

C and D have access to the current x; E and A to both x and y.



The overlap between the two is the trap, if you try to assign to the
same variable in the loop header and then update it in the loop body.

Not to mention the inconsistency that some assignments are accessible
from the surrounding code:

 [expr for a in (x := func(), ...) ]
 print(x)  # works

while the most useful ones, those in the body, will be locked up in an
implicit sublocal scope where they are unreachable from outside of the
comprehension:

 [x := something ...  for a in sequence ]
 print(x)  # fails




--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Steven D'Aprano wrote:
The *very first* motivating example for this proposal came from a 
comprehension.


I think it is both unfortunate and inevitable that the discussion bogged
down in comprehension-hell.


I think the unfortunateness started when we crossed over from
talking about binding a temporary name for use *within* a
comprehension or expression, to binding a name for use *outside*
the comprehension or expression where it's bound.

As long as it's for internal use, whether it's in a comprehension
or not isn't an issue.

Tim Peters has also given a 
couple of good examples of mathematical code that would benefit strongly 
from this feature.


Going back a few months now, they were the examples that tipped me over


Well, I remain profoundly unconvinced that writing comprehensions
with side effects is ever a good idea, and Tim's examples did
nothing to change that.

--
Greg

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Eric Fahlgren]

On Wed, Jun 27, 2018 at 9:27 AM Paul Moore  wrote:

> From my reading, PEP 572 takes the position that "parent local
> scoping" is what people expect from assignment expressions *in
> comprehensions* and it's useful enough that there is no reason not to
> make that the behaviour. The behaviour isn't generally useful enough
> to be worth exposing as a primitive (it's not even useful enough for
> the PEP to give it an explicit name!) so it's just a special case for
> assignment expressions in comprehensions/generators.
>

​So, my interpretation is that it will behave like this?

x = 2
y = [x := 3 for i in range(1)]
print(x)
3

def f():
x = 4
y = [x := 5 for i in range(1)]
print(x)
f()
5

class C:
x = 6
y = [x := 7 for i in range(1)]
print(x)
C()
6
print(x)
7​
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Intent to accept PEP 561 -- Distributing and Packaging Type Information

[Ivan Levkivskyi]

Congrats Ethan,

Well done! I think PEP 561 will significantly simplify typing third party
modules.

--
Ivan



On 28 June 2018 at 00:11, Guido van Rossum  wrote:

> Well, with that, I am hereby accepting PEP 561.
>
> Ethan has done a tremendous job writing this PEP and implementing it, and
> I am sure that package and stub authors will be very glad to hear that
> there are now officially supported ways other than typeshed to distribute
> type annotations.
>
> Congrats Ethan!
>
> --Guido
>
> On Mon, Jun 25, 2018 at 12:15 PM Guido van Rossum 
> wrote:
>
>> OK, last call! I'll accept the current draft tomorrow unless someone
>> pushes back.
>>
>> On Fri, Jun 22, 2018 at 8:37 AM Nick Coghlan  wrote:
>>
>>> On 23 June 2018 at 01:16, Guido van Rossum  wrote:
>>> > That sounds like you're supporting PEP 561 as is, right?
>>>
>>> Aye, I'm personally fine with it - we do need to do something about
>>> automatically reserving the derived names on PyPI, but I don't think
>>> that's a blocker for the initial PEP acceptance (instead, it will go
>>> the other way: PEP acceptance will drive Warehouse getting updated to
>>> handle the convention already being adopted by the client tools).
>>>
>>> > Excuse my
>>> > ignorance, but where are API testing stub interfaces described or used?
>>>
>>> They're not - it's just the context for Donald referring to "stubs" as
>>> being a general technical term with other meanings beyond the "type
>>> hinting stub file" one.
>>>
>>> As such, there's three parts to explaining why we're not worried about
>>> the terminology clash:
>>>
>>> - Ethan searched for projects called "*-stubs" or "*_stubs" and didn't
>>> find any, so the practical impact of any terminology clash will be low
>>> - there isn't an established need to automatically find testing stub
>>> libraries based on an existing project name the way there is for type
>>> hints
>>> - even if such a need did arise in the future, the "py.typed" marker
>>> file and the different file extension for stub files within a package
>>> still gives us an enormous amount of design flexibility
>>>
>>> Cheers,
>>> Nick.
>>>
>>> --
>>> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> levkivskyi%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Intent to accept PEP 561 -- Distributing and Packaging Type Information

[Guido van Rossum]

Well, with that, I am hereby accepting PEP 561.

Ethan has done a tremendous job writing this PEP and implementing it, and I
am sure that package and stub authors will be very glad to hear that there
are now officially supported ways other than typeshed to distribute type
annotations.

Congrats Ethan!

--Guido

On Mon, Jun 25, 2018 at 12:15 PM Guido van Rossum  wrote:

> OK, last call! I'll accept the current draft tomorrow unless someone
> pushes back.
>
> On Fri, Jun 22, 2018 at 8:37 AM Nick Coghlan  wrote:
>
>> On 23 June 2018 at 01:16, Guido van Rossum  wrote:
>> > That sounds like you're supporting PEP 561 as is, right?
>>
>> Aye, I'm personally fine with it - we do need to do something about
>> automatically reserving the derived names on PyPI, but I don't think
>> that's a blocker for the initial PEP acceptance (instead, it will go
>> the other way: PEP acceptance will drive Warehouse getting updated to
>> handle the convention already being adopted by the client tools).
>>
>> > Excuse my
>> > ignorance, but where are API testing stub interfaces described or used?
>>
>> They're not - it's just the context for Donald referring to "stubs" as
>> being a general technical term with other meanings beyond the "type
>> hinting stub file" one.
>>
>> As such, there's three parts to explaining why we're not worried about
>> the terminology clash:
>>
>> - Ethan searched for projects called "*-stubs" or "*_stubs" and didn't
>> find any, so the practical impact of any terminology clash will be low
>> - there isn't an established need to automatically find testing stub
>> libraries based on an existing project name the way there is for type
>> hints
>> - even if such a need did arise in the future, the "py.typed" marker
>> file and the different file extension for stub files within a package
>> still gives us an enormous amount of design flexibility
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 05:52:16PM +0300, Ivan Pozdeev via Python-Dev wrote:

> >What this means in practice is that assignments will go to different
> >scopes depending on *where* they are in the comprehension:
> >
> > [ expr   for x in iter1  for y in iter2  if cond   ...]
> > [ BB for x in AA for y in BB if BB ...]
> >
> >Assignments in the section marked "AA" will be in the local scope;
> >assignments in the BB sections will be in the sublocal scope. That's
> >not too bad, up to the point you try to assign to the same name in
> >AA and BB. And then you are likely to get confusing hard to
> >debug UnboundLocalErrors.
> 
> This isn't as messy as you make it sound if you remember that the 
> outermost iterable is evaluated only once at the start and all the 
> others -- each iteration.

The question isn't *how often* they are evaluated, or how many loops you 
have, but *what scope* they are evaluated in. Even in a single loop 
comprehension, parts of it are evaluated in the local scope and parts 
are evaluated in an implicit sublocal scope.

The overlap between the two is the trap, if you try to assign to the 
same variable in the loop header and then update it in the loop body.

Not to mention the inconsistency that some assignments are accessible 
from the surrounding code:

[expr for a in (x := func(), ...) ]
print(x)  # works

while the most useful ones, those in the body, will be locked up in an 
implicit sublocal scope where they are unreachable from outside of the 
comprehension:

[x := something ...  for a in sequence ]
print(x)  # fails


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Guido van Rossum]

So IIUC you are okay with the behavior described by the PEP but you want an
explicit language feature to specify it?

I don't particularly like adding a `parentlocal` statement to the language,
because I don't think it'll be generally useful. (We don't have `goto` in
the language even though it could be used in the formal specification of
`if`, for example. :-)

But as a descriptive mechanism to make the PEP's spec clearer I'm fine with
it. Let's call it `__parentlocal` for now. It would work a bit like
`nonlocal` but also different, since in the normal case (when there's no
matching `nonlocal` in the parent scope) it would make the target a local
in that scope rather than trying to look for a definition of the target
name in surrounding (non-class, non-global) scopes. Also if there's a
matching `global` in the parent scope, `__parentlocal` itself changes its
meaning to `global`. If you want to push a target through several level of
target scopes you can do that by having a `__parentlocal` in each scope
that it should push through (this is needed for nested comprehensions, see
below).

Given that definition of `__parentlocal`, in first approximation the
scoping rule proposed by PEP 572 would then be: In comprehensions (which in
my use in the PEP 572 discussion includes generator expressions) the
targets of inline assignments are automatically endowed with a
`__parentlocal` declaration, except inside the "outermost iterable" (since
that already runs in the parent scope).

There would have to be additional words when comprehensions themselves are
nested (e.g. `[[a for a in range(i)] for i in range(10)]`) since the PEP's
intention is that inline assignments anywhere there end up targeting the
scope containing the outermost comprehension. But this can all be expressed
by adding `__parentlocal` for various variables in various places
(including in the "outermost iterable" of inner comprehensions).

I'd also like to keep the rule prohibiting use of the same name as a
comprehension loop control variable and as an inline assignment target;
this rule would also prohibit shenanigans with nested comprehensions (for
any set of nested comprehensions, any name that's a loop control variable
in any of them cannot be an inline assignment target in any of them). This
would also apply to the "outermost iterable".

Does this help at all, or did I miss something?

--Guido

On Wed, Jun 27, 2018 at 5:27 AM Nick Coghlan  wrote:

> On 26 June 2018 at 02:27, Guido van Rossum  wrote:
> > [This is my one reply in this thread today. I am trying to limit the
> amount
> > of time I spend to avoid another overheated escalation.]
>
> Aye, I'm trying to do the same, and deliberately spending some
> evenings entirely offline is helping with that :)
>
> > On Mon, Jun 25, 2018 at 4:44 AM Nick Coghlan  wrote:
> >>
> >> Right, the proposed blunt solution to "Should I use 'NAME = EXPR' or
> >> 'NAME := EXPR'?" bothers me a bit, but it's the implementation
> >> implications of parent local scoping that I fear will create a
> >> semantic tar pit we can't get out of later.
> >
> > Others have remarked this too, but it really bother me that you are
> focusing
> > so much on the implementation of parent local scoping rather than on the
> > "intuitive" behavior which is super easy to explain -- especially to
> someone
> > who isn't all that familiar (or interested) with the implicit scope
> created
> > for the loop control variable(s). According to Steven (who noticed that
> this
> > is barely mentioned in most tutorials about comprehensions) that is most
> > people, however very few of them read python-dev.
> >
> > It's not that much work for the compiler, since it just needs to do a
> little
> > bit of (new) static analysis and then it can generate the bytecode to
> > manipulate closure(s). The runtime proper doesn't need any new
> > implementation effort. The fact that sometimes a closure must be
> introduced
> > where no explicit initialization exists is irrelevant to the runtime --
> this
> > only affects the static analysis, at runtime it's no different than if
> the
> > explicit initialization was inside `if 0`.
>
> One of the things I prize about Python's current code generator is how
> many of the constructs can be formulated as simple content-and-context
> independent boilerplate removal, which is why parent local scoping (as
> currently defined in PEP 572) bothers me: rather than being a new
> primitive in its own right, the PEP instead makes the notion of "an
> assignment expression in a comprehension or generator expression" a
> construct that can't readily decomposed into lower level building
> blocks the way that both assignment expressions on their own and
> comprehensions and generator expressions on their own can be. Instead,
> completely new language semantics arise from the interaction between
> two otherwise independent features.
>
> Even changes as complicated as PEP 343's with statement, PEP 380's
> yield from, and PEP 492's 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Eric V. Smith]

> On Jun 27, 2018, at 9:49 AM, Steven D'Aprano  wrote:
> 
>> On Wed, Jun 27, 2018 at 08:00:20AM -0400, Eric V. Smith wrote:
>>> On 6/27/2018 7:08 AM, Chris Angelico wrote:
>>> It gets funnier with nested loops. Or scarier. I've lost the ability
>>> to distinguish those two.
>>> 
>>> def test():
>>>spam = 1
>>>ham = 2
>>>vars = [key1+key2 for key1 in locals() for key2 in locals()]
>>>return vars
>>> 
>>> Wanna guess what that's gonna return?
>> 
>> I'm not singling out Chris here, but these discussions would be easier 
>> to follow and more illuminating if the answers to such puzzles were 
>> presented when they're posed.
> 
> You can just copy and paste the function into the interactive 
> interpreter and run it :-)

Not on my phone when I’m riding a bus, I can’t. I’m trying to more or less 
follow the discussion, but the “guess what this will do” aspect of the 
discussion makes it hard.

Eric 

> 
> But where's the fun in that? The point of the exercise is to learn first 
> hand just how complicated it is to try to predict the *current* scope 
> behaviour of comprehensions. Without the ability to perform assignment 
> inside them, aside from the loop variable, we've managed to avoid 
> thinking too much about this until now.
> 
> It also demonstrates the unrealisticness of treating comprehensions as a 
> separate scope -- they're hybrid scope, with parts of the comprehension 
> running in the surrounding local scope, and parts running in an sublocal 
> scope.
> 
> Earlier in this thread, Nick tried to justify the idea that 
> comprehensions run in their own scope, no matter how people think of 
> them -- but that's an over-simplification, as Chris' example above 
> shows. Parts of the comprehension do in fact behave exactly as the naive 
> model would suggest (even if Nick is right that other parts don't).
> 
> As complicated and hairy as the above example is, (1) it is a pretty 
> weird thing to do, so most of us will almost never need to consider it; 
> and (2) backwards compatibility requires that we live with it now (at 
> least unless we introduce a __future__ import).
> 
> If we can't simplify the scope of comprehensions, we can at least 
> simplify the parts that actually matters. What matters are the loop 
> variables (already guaranteed to be sublocal and not "leak" out of the 
> comprehension) and the behaviour of assignment expressions (open to 
> discussion).
> 
> Broadly speaking, there are two positions we can take:
> 
> 1. Let the current implementation of comprehensions as an implicit 
> hidden function drive the functionality; that means we duplicate the 
> hairiness of the locals() behaviour seen above, although it won't be 
> obvious at first glance.
> 
> What this means in practice is that assignments will go to different 
> scopes depending on *where* they are in the comprehension:
> 
>[ expr   for x in iter1  for y in iter2  if cond   ...]
>[ BB for x in AA for y in BB if BB ...]
> 
> Assignments in the section marked "AA" will be in the local scope; 
> assignments in the BB sections will be in the sublocal scope. That's 
> not too bad, up to the point you try to assign to the same name in 
> AA and BB. And then you are likely to get confusing hard to 
> debug UnboundLocalErrors.
> 
> 
> 2. Or we can keep the current behaviour for locals and the loop 
> variables, but we can keep assignment expressions simple by ensuring 
> they always bind to the enclosing scope. Compared to the complexity of 
> the above, we have the relatively straight forward:
> 
>[ AA for x in AA for y in AA if AA ...]
> 
> The loop variables continue to be hidden away in the invisible, implicit 
> comprehension function, where they can't leak out, while explicit 
> assignments to variables (using := or given or however it is spelled) 
> will always go into the surrounding local scope, like they do in every 
> other expression.
> 
> Does it matter that the implementation of this requires an implicit 
> nonlocal declaration for each assignment? No more than it matters that 
> comprehensions themselves require an implicit function.
> 
> And what we get out of this is simpler semantics at the Python level:
> 
> - Unless previous declared global, assignment expressions always bind to 
> the current scope, even if they're inside a comprehension;
> 
> - and we don't have to deal with the oddity that different bits of a 
> comprehension run in different scopes (unless we go out of our way to 
> use locals()); merely using assignment expressions will just work 
> consistently and simply, and loop variables will still be confined to 
> the comprehension as they are now.
> 
> 
> -- 
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

Re: [Python-Dev] Policy on refactoring/clean up

[Kyle]

In itself, the code clean up you have done is a good thing in the sense
that you re-organized things and in my understanding, they look good now.
In some of the teams I've been granted to work, there was a rule stating
that whenever a dev would work on an enhancement/bugfix, he would create a
separate ticket that would track all code refactoring related to his work.
The strategy is quite different here and the suggestion was to have your
refactoring be part of an enhancement/bugfix. The most valuable argument in
favor of all the current reviewers is Guido van Rossum's comment on the
effect of having git blame made harder.
For some of the projects I'm currently working on, we have code cleanup
teams where members are affected expressly to refactoring code. We might
end up needing such things in the future but for now, it is wiser to leave
the code as is and focus on fixing actual problems.

-- 
*Q::Drone's Co-Founder and Co-CEO*
 *Hervé "Kyle" MUTOMBO*

*Envoyé depuis le web.*
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Nick Coghlan]

> However, PEP 572 in its current form takes the position "parent local
> scoping is sufficiently useful to make it a required pre-requisite for
> adding assignment expressions, but not useful enough to expose as a
> new scope declaration primitive",
>

Of course the PEP doesn't take that position at all:  it doesn't even
contain the term "parent local scoping".  That's your term, which nobody
else uses unless they're replying to you ;-)

What the PEP does say:

"""
an assignment expression occurring in a list, set or dict comprehension or
in a generator expression (below collectively referred to as
"comprehensions") binds the target in the containing scope, honoring a
nonlocal or global declaration for the target in that scope, if one exists.
For the purpose of this rule the containing scope of a nested comprehension
is the scope that contains the outermost comprehension. A lambda counts as
a containing scope.
"""

It's a small collection of plainly stated rules for specifying the intended
semantics.  If you want to claim that this _is_ "useful enough to expose as
a new scope declaration primitive", it's really on you to present use cases
to justify that claim.  I'd present some for you, but I don't have any (I
don't care that "by hand" conversion of nested comprehensions to workalike
Python nested functions may require a bit of thought to establish the
intended scope of assignment expression target names - all of which is
easily doable without adding any new statements).

I don't _expect_ that other good use cases exist.  The gimmick's purpose is
to make code that visually _appears_ to belong to a block act as if
embedded assignments do occur in that block.  If there's an explicitly
nested function, that fundamental motivation no longer applies.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Nick Coghlan]>

>  actually made those semantics available as an explicit

> "parentlocal NAME" declaration ...:
> >
> > def _list_comp(_outermost_iter):
> > parentlocal item
> > _result = []
> > for x in _outermost_iter:
> > item = x
> > _result.append(x)
> > return _result
> >
> > _expr_result = _list_comp(items)
>

[Greg Ewing]

I'm not sure that's possible. If I understand correctly,
> part of the definition of "parent local" is that "parent"
> refers to the nearest enclosing *non-comprehension* scope,
> to give the expected result for nested comprehensions.
> If that's so, then it's impossible to fully decouple its
> definition from comprehensions.
>
>  Nick's "parentlocal" does refer to the parent, but makes no distinction
between synthesized and user-written functions.  If the parent has a
matching parentlocal declaration for the same name then the original really
refers to the grandparent - and so on.  Ultimately, it resolves to the
closest enclosing scope in which the name is _not_ declared parentlocal.
In that scope, a "nonlocal" or "global" declaration settles it if one
appears, else the name is local to that scope.

So a nested comprehension would declare its assignment expression targets
as parentlocal in its synthesized function, and in all the containing
synthesized functions generated for containing comprehensions.

This appears in some strained ;-) way "natural" only because there is no
explicit way to declare something "local" in Python.  In just about any
other language with closures and nested lexical scopes, comprehensions and
generator expressions would have been implemented via nested functions that
explicitly declared their "for" target names "local". and nothing else.
The only change needed then for PEP 572 (b) semantics would be to declare
assignment expression target names local (if their scope wasn't already
known) in the closest containing non-synthesized block.

None of which really matters.  The real question is which semantics are
desired.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Paul Moore]

On 27 June 2018 at 15:39, Nick Coghlan  wrote:

> However, PEP 572 in its current form takes the position "parent local
> scoping is sufficiently useful to make it a required pre-requisite for
> adding assignment expressions, but not useful enough to expose as a
> new scope declaration primitive", and I've come to the view that it
> really is the "A+B=MAGIC!" aspect of the current proposal that bothers
> me, whereas "A+B implies C for " doesn't bother me
> any more than the implicit non-local references introduced as part of
> the original lexical scoping changes bother me.

>From my reading, PEP 572 takes the position that "parent local
scoping" is what people expect from assignment expressions *in
comprehensions* and it's useful enough that there is no reason not to
make that the behaviour. The behaviour isn't generally useful enough
to be worth exposing as a primitive (it's not even useful enough for
the PEP to give it an explicit name!) so it's just a special case for
assignment expressions in comprehensions/generators.

That seems to me like a classic example of practicality beating purity.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 27.06.2018 16:49, Steven D'Aprano wrote:

On Wed, Jun 27, 2018 at 08:00:20AM -0400, Eric V. Smith wrote:

On 6/27/2018 7:08 AM, Chris Angelico wrote:

It gets funnier with nested loops. Or scarier. I've lost the ability
to distinguish those two.

def test():
 spam = 1
 ham = 2
 vars = [key1+key2 for key1 in locals() for key2 in locals()]
 return vars

Wanna guess what that's gonna return?

I'm not singling out Chris here, but these discussions would be easier
to follow and more illuminating if the answers to such puzzles were
presented when they're posed.

You can just copy and paste the function into the interactive
interpreter and run it :-)

But where's the fun in that? The point of the exercise is to learn first
hand just how complicated it is to try to predict the *current* scope
behaviour of comprehensions. Without the ability to perform assignment
inside them, aside from the loop variable, we've managed to avoid
thinking too much about this until now.

It also demonstrates the unrealisticness of treating comprehensions as a
separate scope -- they're hybrid scope, with parts of the comprehension
running in the surrounding local scope, and parts running in an sublocal
scope.

Earlier in this thread, Nick tried to justify the idea that
comprehensions run in their own scope, no matter how people think of
them -- but that's an over-simplification, as Chris' example above
shows. Parts of the comprehension do in fact behave exactly as the naive
model would suggest (even if Nick is right that other parts don't).

As complicated and hairy as the above example is, (1) it is a pretty
weird thing to do, so most of us will almost never need to consider it;
and (2) backwards compatibility requires that we live with it now (at
least unless we introduce a __future__ import).

If we can't simplify the scope of comprehensions, we can at least
simplify the parts that actually matters. What matters are the loop
variables (already guaranteed to be sublocal and not "leak" out of the
comprehension) and the behaviour of assignment expressions (open to
discussion).

Broadly speaking, there are two positions we can take:

1. Let the current implementation of comprehensions as an implicit
hidden function drive the functionality; that means we duplicate the
hairiness of the locals() behaviour seen above, although it won't be
obvious at first glance.

What this means in practice is that assignments will go to different
scopes depending on *where* they are in the comprehension:

 [ expr   for x in iter1  for y in iter2  if cond   ...]
 [ BB for x in AA for y in BB if BB ...]

Assignments in the section marked "AA" will be in the local scope;
assignments in the BB sections will be in the sublocal scope. That's
not too bad, up to the point you try to assign to the same name in
AA and BB. And then you are likely to get confusing hard to
debug UnboundLocalErrors.


This isn't as messy as you make it sound if you remember that the 
outermost iterable is evaluated only once at the start and all the 
others -- each iteration.

Anyone using comprehensions has to know this fact.
The very readable syntax also makes it rather straightforward (though 
admittedly requiring some hand-tracing) to figure out what is evaluated 
after what.




2. Or we can keep the current behaviour for locals and the loop
variables, but we can keep assignment expressions simple by ensuring
they always bind to the enclosing scope. Compared to the complexity of
the above, we have the relatively straight forward:

 [ AA for x in AA for y in AA if AA ...]

The loop variables continue to be hidden away in the invisible, implicit
comprehension function, where they can't leak out, while explicit
assignments to variables (using := or given or however it is spelled)
will always go into the surrounding local scope, like they do in every
other expression.

Does it matter that the implementation of this requires an implicit
nonlocal declaration for each assignment? No more than it matters that
comprehensions themselves require an implicit function.

And what we get out of this is simpler semantics at the Python level:

- Unless previous declared global, assignment expressions always bind to
the current scope, even if they're inside a comprehension;

- and we don't have to deal with the oddity that different bits of a
comprehension run in different scopes (unless we go out of our way to
use locals()); merely using assignment expressions will just work
consistently and simply, and loop variables will still be confined to
the comprehension as they are now.




--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 27 June 2018 at 23:38, Greg Ewing  wrote:
> Nick Coghlan wrote:
>>
>> actually made those semantics available as an explicit
>> "parentlocal NAME" declaration ...:
>>
>> def _list_comp(_outermost_iter):
>> parentlocal item
>> _result = []
>> for x in _outermost_iter:
>> item = x
>> _result.append(x)
>> return _result
>>
>> _expr_result = _list_comp(items)
>
>
> I'm not sure that's possible. If I understand correctly,
> part of the definition of "parent local" is that "parent"
> refers to the nearest enclosing *non-comprehension* scope,
> to give the expected result for nested comprehensions.
> If that's so, then it's impossible to fully decouple its
> definition from comprehensions.

I'm OK with a target scope declaration construct having
lexical-scope-dependent behaviour - exactly what "nonlocal NAME" will
do depends on both the nature of the current scope, and on which names
are declared as local in which outer scopes, and that's also
implicitly the case for all name lookups.

However, PEP 572 in its current form takes the position "parent local
scoping is sufficiently useful to make it a required pre-requisite for
adding assignment expressions, but not useful enough to expose as a
new scope declaration primitive", and I've come to the view that it
really is the "A+B=MAGIC!" aspect of the current proposal that bothers
me, whereas "A+B implies C for " doesn't bother me
any more than the implicit non-local references introduced as part of
the original lexical scoping changes bother me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python and Linux Standard Base

[Nick Coghlan]

On 27 June 2018 at 23:57, Charalampos Stratakis  wrote:
> From: "Antoine Pitrou" 
>> One question: who maintains the LSB?
>>
>> The fact that the Python portion was never updated may hint that nobody
>> uses it...
>
> That could definitely be the case here. I stumbled upon that when checking 
> shebang requirements on Fedora
> and apparently every distro has a sort of meta package that adheres to those 
> standards. In Fedora's case [0].
>
> I don't have a good answer on who maintains it or even how compliant some 
> distros are, but I was wondering
> if that topic came up beforehand and if any requirements were placed from 
> either side.

My impression while working for Red Hat was that LSB ended up being
one of those bureaucratic standards that ended up sprawling so far
beyond being a minimal system, while still leaving core capabilities
that real world apps rely on underspecified, that compatibility and
compliance testing became sufficiently painful that folks that cared
about certifications started certifying a handful of major stable
distros instead (with a common modern selection being Ubuntu LTS,
Debian Stable, RHEL/CentOS, and SLES).

https://en.wikipedia.org/wiki/Linux_Standard_Base seems to back up
that impression, with neither Debian nor Ubuntu claiming LSB support
at all these days.

Given the rise of Flatpak, Snappy, and Linux containers in general, it
may make sense to suggest that LSB drop Python entirely (similar to
what they did for Java, albeit for different reasons), and instead
recommend that portable applications requiring Python bundle their own
interpreter.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 27.06.2018 16:25, Greg Ewing wrote:

Ivan Pozdeev via Python-Dev wrote:
Using this assigned result elsewhere in the same expression (akin to 
regex backreferences) is not a part of the basic idea actually.


If that's true, then the proposal has mutated into something
that has *no* overlap whatsoever with the use case that started
this whole discussion,


I don't know what and where "started" it (AFAIK the idea has been around 
for years) but for me, the primary use case for an assignment expression 
is to be able to "catch" a value into a variable in places where I can't 
put an assignment statement in, like the infamous `if re.match() is not 
None'.



which was about binding a temporary
variable in a comprehension, for use *within* the comprehension.


Then I can't understand all the current fuss about scoping.
AFAICS, it's already like I described in 
https://mail.python.org/pipermail/python-dev/2018-June/154067.html :
the outermost iterable is evaluated in the local scope while others in 
the internal one:


In [13]: [(l,i) for l in list(locals())[:5] for i in locals()]
Out[13]:
[('__name__', 'l'),
 ('__name__', '.0'),
 ('__builtin__', 'l'),
 ('__builtin__', '.0'),
 ('__builtin__', 'i'),
 ('__builtins__', 'l'),
 ('__builtins__', '.0'),
 ('__builtins__', 'i'),
 ('_ih', 'l'),
 ('_ih', '.0'),
 ('_ih', 'i'),
 ('_oh', 'l'),
 ('_oh', '.0'),
 ('_oh', 'i')]

(note that `i' is bound after the first evaluation of internal 
`locals()' btw, as to be expected)


If the "temporary variables" are for use inside the comprehension only, 
the assignment expression needs to bind in the current scope like the 
regular assignment statement, no changes are needed!


It depends on the evaluation order (and whether something is 
evaluated at all),


Which to my mind is yet another reason not to like ":=".



--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 03:41:23PM +0200, Antoine Pitrou wrote:
> 
> Why is this discussion talking about comprehensions at all?
> Is there a decent use case for using assignments in comprehensions (as
> opposed to language lawyering or deliberate obfuscation)?

Yes. The *very first* motivating example for this proposal came from a 
comprehension.

I think it is both unfortunate and inevitable that the discussion bogged
down in comprehension-hell. Unfortunate because I don't think that the 
most compelling use-cases involve comprehensions at all. But inevitable 
because *comprehensions are the hard case*, thanks to the (justifiable!) 
decision to implement them as implicit hidden functions.

In my opinion, the really two BIG wins for assignment expressions are 
while loops and cascades of if... blocks. Tim Peters has also given a 
couple of good examples of mathematical code that would benefit strongly 
from this feature.

Going back a few months now, they were the examples that tipped me over 
from the opinion

"Oh, just re-write the comprehension as a loop"

to the opinion

"You know, I think this feature actually is useful... and
as a bonus, you can keep using the comprehension"

But that requires that we get the comprehension scoping right. Not just 
leave it as an unspecified implementation detail.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python and Linux Standard Base

[Charalampos Stratakis]

- Original Message -
> From: "Antoine Pitrou" 
> To: python-dev@python.org
> Sent: Wednesday, June 27, 2018 3:42:53 PM
> Subject: Re: [Python-Dev] Python and Linux Standard Base
> 
> On Wed, 27 Jun 2018 09:18:24 -0400 (EDT)
> Charalampos Stratakis  wrote:
> > 
> > My question is, if there is any incentive to try and ask for
> > modernization/amendment  of the standards?
> > I really doubt that any linux distro at that point can be considered lsb
> > compliant at least from the
> > python side of things.
> 
> One question: who maintains the LSB?
> 
> The fact that the Python portion was never updated may hint that nobody
> uses it...
> 
> Regards
> 
> Antoine.
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/cstratak%40redhat.com
> 

That could definitely be the case here. I stumbled upon that when checking 
shebang requirements on Fedora
and apparently every distro has a sort of meta package that adheres to those 
standards. In Fedora's case [0].

I don't have a good answer on who maintains it or even how compliant some 
distros are, but I was wondering
if that topic came up beforehand and if any requirements were placed from 
either side.

[0] 
https://src.fedoraproject.org/rpms/redhat-lsb/blob/master/f/redhat-lsb.spec#_419


-- 
Regards,

Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python and Linux Standard Base

[David Mertz]

The main wiki page was last touched at all in 2016. The mailing list in Jan
2018 had about 8 comments, none of them actually related to LSB. They
stopped archiving the ML altogether in Feb 2018. I think it's safe to say
the parrot is dead.

On Wed, Jun 27, 2018, 9:50 AM Antoine Pitrou  wrote:

> On Wed, 27 Jun 2018 09:18:24 -0400 (EDT)
> Charalampos Stratakis  wrote:
> >
> > My question is, if there is any incentive to try and ask for
> modernization/amendment  of the standards?
> > I really doubt that any linux distro at that point can be considered lsb
> compliant at least from the
> > python side of things.
>
> One question: who maintains the LSB?
>
> The fact that the Python portion was never updated may hint that nobody
> uses it...
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/mertz%40gnosis.cx
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 08:00:20AM -0400, Eric V. Smith wrote:
> On 6/27/2018 7:08 AM, Chris Angelico wrote:
> >It gets funnier with nested loops. Or scarier. I've lost the ability
> >to distinguish those two.
> >
> >def test():
> > spam = 1
> > ham = 2
> > vars = [key1+key2 for key1 in locals() for key2 in locals()]
> > return vars
> >
> >Wanna guess what that's gonna return?
> 
> I'm not singling out Chris here, but these discussions would be easier 
> to follow and more illuminating if the answers to such puzzles were 
> presented when they're posed.

You can just copy and paste the function into the interactive 
interpreter and run it :-)

But where's the fun in that? The point of the exercise is to learn first 
hand just how complicated it is to try to predict the *current* scope 
behaviour of comprehensions. Without the ability to perform assignment 
inside them, aside from the loop variable, we've managed to avoid 
thinking too much about this until now.

It also demonstrates the unrealisticness of treating comprehensions as a 
separate scope -- they're hybrid scope, with parts of the comprehension 
running in the surrounding local scope, and parts running in an sublocal 
scope.

Earlier in this thread, Nick tried to justify the idea that 
comprehensions run in their own scope, no matter how people think of 
them -- but that's an over-simplification, as Chris' example above 
shows. Parts of the comprehension do in fact behave exactly as the naive 
model would suggest (even if Nick is right that other parts don't).

As complicated and hairy as the above example is, (1) it is a pretty 
weird thing to do, so most of us will almost never need to consider it; 
and (2) backwards compatibility requires that we live with it now (at 
least unless we introduce a __future__ import).

If we can't simplify the scope of comprehensions, we can at least 
simplify the parts that actually matters. What matters are the loop 
variables (already guaranteed to be sublocal and not "leak" out of the 
comprehension) and the behaviour of assignment expressions (open to 
discussion).

Broadly speaking, there are two positions we can take:

1. Let the current implementation of comprehensions as an implicit 
hidden function drive the functionality; that means we duplicate the 
hairiness of the locals() behaviour seen above, although it won't be 
obvious at first glance.

What this means in practice is that assignments will go to different 
scopes depending on *where* they are in the comprehension:

[ expr   for x in iter1  for y in iter2  if cond   ...]
[ BB for x in AA for y in BB if BB ...]

Assignments in the section marked "AA" will be in the local scope; 
assignments in the BB sections will be in the sublocal scope. That's 
not too bad, up to the point you try to assign to the same name in 
AA and BB. And then you are likely to get confusing hard to 
debug UnboundLocalErrors.


2. Or we can keep the current behaviour for locals and the loop 
variables, but we can keep assignment expressions simple by ensuring 
they always bind to the enclosing scope. Compared to the complexity of 
the above, we have the relatively straight forward:

[ AA for x in AA for y in AA if AA ...]

The loop variables continue to be hidden away in the invisible, implicit 
comprehension function, where they can't leak out, while explicit 
assignments to variables (using := or given or however it is spelled) 
will always go into the surrounding local scope, like they do in every 
other expression.

Does it matter that the implementation of this requires an implicit 
nonlocal declaration for each assignment? No more than it matters that 
comprehensions themselves require an implicit function.

And what we get out of this is simpler semantics at the Python level:

- Unless previous declared global, assignment expressions always bind to 
the current scope, even if they're inside a comprehension;

- and we don't have to deal with the oddity that different bits of a 
comprehension run in different scopes (unless we go out of our way to 
use locals()); merely using assignment expressions will just work 
consistently and simply, and loop variables will still be confined to 
the comprehension as they are now.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python and Linux Standard Base

[Antoine Pitrou]

On Wed, 27 Jun 2018 09:18:24 -0400 (EDT)
Charalampos Stratakis  wrote:
> 
> My question is, if there is any incentive to try and ask for 
> modernization/amendment  of the standards?
> I really doubt that any linux distro at that point can be considered lsb 
> compliant at least from the
> python side of things.

One question: who maintains the LSB?

The fact that the Python portion was never updated may hint that nobody
uses it...

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Antoine Pitrou]

Why is this discussion talking about comprehensions at all?
Is there a decent use case for using assignments in comprehensions (as
opposed to language lawyering or deliberate obfuscation)?

Regards

Antoine.


On Thu, 28 Jun 2018 01:25:14 +1200
Greg Ewing  wrote:
> Ivan Pozdeev via Python-Dev wrote:
> > Using this assigned result elsewhere in the same expression (akin to 
> > regex backreferences) is not a part of the basic idea actually.  
> 
> If that's true, then the proposal has mutated into something
> that has *no* overlap whatsoever with the use case that started
> this whole discussion, which was about binding a temporary
> variable in a comprehension, for use *within* the comprehension.
> 
> > It depends on the evaluation order (and whether something is evaluated 
> > at all),  
> 
> Which to my mind is yet another reason not to like ":=".
> 



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Nick Coghlan wrote:

actually made those semantics available as an explicit
"parentlocal NAME" declaration ...:

def _list_comp(_outermost_iter):
parentlocal item
_result = []
for x in _outermost_iter:
item = x
_result.append(x)
return _result

_expr_result = _list_comp(items)


I'm not sure that's possible. If I understand correctly,
part of the definition of "parent local" is that "parent"
refers to the nearest enclosing *non-comprehension* scope,
to give the expected result for nested comprehensions.
If that's so, then it's impossible to fully decouple its
definition from comprehensions.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Ivan Pozdeev via Python-Dev wrote:
Using this assigned result elsewhere in the same expression (akin to 
regex backreferences) is not a part of the basic idea actually.


If that's true, then the proposal has mutated into something
that has *no* overlap whatsoever with the use case that started
this whole discussion, which was about binding a temporary
variable in a comprehension, for use *within* the comprehension.

It depends on the evaluation order (and whether something is evaluated 
at all),


Which to my mind is yet another reason not to like ":=".

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python and Linux Standard Base

[Charalampos Stratakis]

LSB (Linux Standard Base) is a set of standards defined from the Linux 
Foundation
for linux distributions [0][1] with the latest version (LSB 5.0) released on 3rd
of June, 2015.

Python is also mentioned there but the information is horribly outdated [2]. 
For example
here are the necessary modules that a python interpreter should include in an 
lsb compliant
system [3] and the minimum python version should be 2.4.2. Also the python3 
interpreter
is never mentioned [4].

My question is, if there is any incentive to try and ask for 
modernization/amendment  of the standards?
I really doubt that any linux distro at that point can be considered lsb 
compliant at least from the
python side of things.

[0] https://en.wikipedia.org/wiki/Linux_Standard_Base
[1] https://wiki.linuxfoundation.org/lsb/lsb-50
[2] 
https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Languages/LSB-Languages/python.html
[3] 
https://refspecs.linuxfoundation.org/LSB_5.0.0/LSB-Languages/LSB-Languages/pymodules.html
[4] https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3677
-- 
Regards,

Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 27.06.2018 5:36, Guido van Rossum wrote:

[This is my one response today]

On Mon, Jun 25, 2018 at 12:40 PM Terry Reedy > wrote:


On 6/24/2018 7:25 PM, Guido van Rossum wrote:
> I'd wager that the people who might be most horrified about it

the (b) scoping rule change

> would be people who feel strongly that the change to the
> comprehension scope rules in Python 3 is a big improvement,

I might not be one of those 'most horrified' by (b), but I
increasingly
don't like it, and I was at best -0 on the comprehension scope
change.
To me, iteration variable assignment in the current scope is a
non-problem.  So to me the change was mostly useless churn. Little
benefit, little harm.  And not worth fighting when others saw a
benefit.


Fair enough, and by itself this might not have been enough reason to 
make the change. But see below.


However, having made the change to nested scopes, I think we should
stick with them.  Or repeal them.  (I believe there is another way to
isolate iteration names -- see  below).  To me, (b) amounts to half
repealing the nested scope change, making comprehensions half-fowl,
half-fish chimeras.


That depends on how you see it -- to me (b) just means that there's an 
implicit nonlocal[1] to make the assignment have the (desirable) 
side-effect.


The key thing to consider here is whether that side-effect is in fact 
desirable. For me, the side-effect of the comprehension's loop control 
variable was never desirable -- it was just an implementation detail 
leaking out. (And that's different from leaking a regular for-loop's 
control variable -- since we have 'break' (and 'else') there are some 
legitimate use cases. But comprehensions try to be expressions, and 
here the side effect is at best useless and at worst a nasty surprise.)


> and who are familiar with the difference in implementation
> of comprehensions (though not generator expressions) in Python 2
vs. 3.

That I pretty much am, I think.  In Python 2, comprehensions (the
fish)
were, at least in effect, expanded in-line to a normal for loop.
Generator expressions (the fowls) were different.  They were, and
still
are, expanded into a temporary generator function whose return
value is
dropped back into the original namespace.  Python 3 turned
comprehensions (with 2 news varieties thereof) into fowls also,
temporary functions whose return value is dropped back in the
original
namespace.  The result is that a list comprehension is equivalent to
list(generator_ expression), even though, for efficiency, it is not
implemented that way.  (To me, this unification is more a benefit
than
name hiding.)


Right, and this consistency convinced me that the change was worth it. 
I just really like to be able to say "[... for ...]" is equivalent to 
"list(... for ...)", and similar for set and dict.


"A shorthand to list()/dict()/set()" is actually how I thought of 
comprehensions when I studied them. And I was actually using list() in 
my code for some time before I learned of their existence.



(b) proposes to add extra hidden code in and around the temporary
function to partly undo the isolation.


But it just adds a nonlocal declaration. There's always some hidden 
code ('def' and 'return' at the very least).


list comprehensions would no
longer be equivalent to list(generator_expression), unless
generator_expressions got the same treatment, in which case they
would
no longer be equivalent to calling the obvious generator function.
Breaking either equivalence might break someone's code.


Ah, there's the rub! I should probably apologize for not clarifying my 
terminology more. In the context of PEP 572, when I say 
"comprehensions" I include generators! PEP 572 states this explicitly 
(https://github.com/python/peps/blame/master/pep-0572.rst#L201-L202).


Certainly PEP 572 intends to add that implicit nonlocal to both 
comprehensions and generator expressions. (I just got really tired of 
writing that phrase over and over, and at some point I forgot that 
this is only a parenthetical remark added in the PEP's latest 
revision, and not conventional terminology -- alas. :-)


Part (b) of PEP 572 does several things of things to *retain* consistency:

- The target of := lives in the same scope regardless of whether it 
occurs in a comprehension, a generator expression, or just in some 
other expression.


- When it occurs in a comprehension or generator expression, the scope 
is the same regardless of whether it occurs in the "outermost 
iterable" or not.


If we didn't have (b) the target would live in the 
comprehension/genexpr scope if it occurred in a comprehension/genexp 
but outside its "outermost iterable", and in the surrounding scope 
otherwise.


---

How loop variables might be isolated without a nested 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 26 June 2018 at 02:27, Guido van Rossum  wrote:
> [This is my one reply in this thread today. I am trying to limit the amount
> of time I spend to avoid another overheated escalation.]

Aye, I'm trying to do the same, and deliberately spending some
evenings entirely offline is helping with that :)

> On Mon, Jun 25, 2018 at 4:44 AM Nick Coghlan  wrote:
>>
>> Right, the proposed blunt solution to "Should I use 'NAME = EXPR' or
>> 'NAME := EXPR'?" bothers me a bit, but it's the implementation
>> implications of parent local scoping that I fear will create a
>> semantic tar pit we can't get out of later.
>
> Others have remarked this too, but it really bother me that you are focusing
> so much on the implementation of parent local scoping rather than on the
> "intuitive" behavior which is super easy to explain -- especially to someone
> who isn't all that familiar (or interested) with the implicit scope created
> for the loop control variable(s). According to Steven (who noticed that this
> is barely mentioned in most tutorials about comprehensions) that is most
> people, however very few of them read python-dev.
>
> It's not that much work for the compiler, since it just needs to do a little
> bit of (new) static analysis and then it can generate the bytecode to
> manipulate closure(s). The runtime proper doesn't need any new
> implementation effort. The fact that sometimes a closure must be introduced
> where no explicit initialization exists is irrelevant to the runtime -- this
> only affects the static analysis, at runtime it's no different than if the
> explicit initialization was inside `if 0`.

One of the things I prize about Python's current code generator is how
many of the constructs can be formulated as simple content-and-context
independent boilerplate removal, which is why parent local scoping (as
currently defined in PEP 572) bothers me: rather than being a new
primitive in its own right, the PEP instead makes the notion of "an
assignment expression in a comprehension or generator expression" a
construct that can't readily decomposed into lower level building
blocks the way that both assignment expressions on their own and
comprehensions and generator expressions on their own can be. Instead,
completely new language semantics arise from the interaction between
two otherwise independent features.

Even changes as complicated as PEP 343's with statement, PEP 380's
yield from, and PEP 492's native coroutines all include examples of
how they could be written *without* the benefit of the new syntax.

By contrast, PEP 572's parent local scoping can't currently be defined
that way. Instead, to explain how the code generator is going to be
expected to handle comprehensions, you have to take the current
comprehension semantics and add two new loops to link up the bound
names correctly::

[item := x for x in items]

becomes:

# Each bound name gets declared as local in the parent scope
if 0:
for item in (): pass
def _list_comp(_outermost_iter):
# Each bound name gets declared as:
#   - nonlocal if outer scope is a function scope
#   - global item if outer scope is a module scope
#   - an error, otherwise
_result = []
for x in _outermost_iter:
_result.append(x)
return _result

_expr_result = _list_comp(items)

This is why my objections would be reduced significantly if the PEP
explicitly admitted that it was defining a new kind of scoping
semantics, and actually made those semantics available as an explicit
"parentlocal NAME" declaration (behind a "from __future__ import
parent_locals" guard), such that the translation of the above example
to an explicitly nested scope could just be the visually
straightforward::

def _list_comp(_outermost_iter):
parentlocal item
_result = []
for x in _outermost_iter:
item = x
_result.append(x)
return _result

_expr_result = _list_comp(items)

That splits up the learning process for anyone trying to really
understand how this particular aspect of Python's code generation
works into two distinct pieces:

- "assignment expressions inside comprehensions and generator
expressions use parent local scoping"
- "parent local scoping works "

If the PEP did that, we could likely even make parent locals work
sensibly for classes by saying that "parent local" for a method
definition in a class body refers to the closure namespace where we
already stash __class__ references for the benefit of zero-arg super
(this would also be a far more robust way of defining private class
variables than name mangling is able to offer).

Having parent locals available as a language level concept (rather
than solely as an interaction between assignment expressions and
implicitly nested scopes) also gets us to a point where
context-independent code thunks that work both at module level and
inside another function can be built as nested 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Eric V. Smith]

On 6/27/2018 7:08 AM, Chris Angelico wrote:

It gets funnier with nested loops. Or scarier. I've lost the ability
to distinguish those two.

def test():
 spam = 1
 ham = 2
 vars = [key1+key2 for key1 in locals() for key2 in locals()]
 return vars

Wanna guess what that's gonna return?


I'm not singling out Chris here, but these discussions would be easier 
to follow and more illuminating if the answers to such puzzles were 
presented when they're posed.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Angelico]

On Wed, Jun 27, 2018 at 7:19 PM, Steven D'Aprano  wrote:
> On Wed, Jun 27, 2018 at 05:52:16PM +1000, Chris Angelico wrote:
>
>> def test():
>> a = 1
>> b = 2
>> vars = {key: locals()[key] for key in locals()}
>> return vars
>>
>> What would your intuition say? Should this be equivalent to dict(locals()) ?
>
> That example is so elegant it makes me want to cry.
>
> And not just because you shadowed the vars() builtin *wink*

It gets funnier with nested loops. Or scarier. I've lost the ability
to distinguish those two.

def test():
spam = 1
ham = 2
vars = [key1+key2 for key1 in locals() for key2 in locals()]
return vars

Wanna guess what that's gonna return?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 576

[Jeroen Demeyer]

On 2018-06-26 21:43, Mark Shannon wrote:

https://github.com/markshannon/pep-576


This actually looks close to Victor Stinner's bpo-29259. But instead of 
storing the function pointer in the class, you're storing it in the 
instance.


One concern that I have is that this might lead to code duplication. You 
require that every class implements its own specialized 
_FOO_FastcallKeywords() function. So you end up with 
_PyCFunction_FastCallKeywords(), _PyMethodDescr_FastCallKeywords(), 
_PyFunction_FastCallKeywords(). If I want to implement a similar class 
myself, I have to reinvent that same wheel again. With PEP 580, I 
replace all those _FOO_FastCallKeywords() functions by one 
PyCCall_FASTCALL() function. Admittedly, my PyCCall_FASTCALL() is more 
complex than each of those _FOO_FastcallKeywords() individually. But 
overall, I think that PEP 580 leads to simpler code.


Second, you still have a performance problem for methods. You made sure 
that the method optimizations in the Python bytecode interpreter 
continue to work, but method calls from C will be slowed down. I don't 
know to what extent and whether it really matters, but it's something to 
note.



Jeroen.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 26.06.2018 1:34, Greg Ewing wrote:

Ivan Pozdeev via Python-Dev wrote:
"as" was suggested even before is became a keyword in `with'. ( if 
(re.match(regex,line) as m) is not None:  )


That's not equivalent where/given, though, since it still
has the asymmetry problem.

What do you mean by "asymmetry"? The fact that the first time around, 
it's the expression and after that, the variable?


If that, it's not a "problem". The whole idea is to assign the result of 
a subexpression to something.
If you force any assignments to be outside, it won't be a subexpression 
anymore, but effectively a separate statement -- if not syntactically, 
then visually at least -- both of which are the things the feature's 
purpose is to avoid.


If you seek to force assignments outside, you should've rather suggested 
inline code blocks e.g. like anonymous methods in C# ( { a=foo(); 
b=bar(); return a+b;} ).


Using this assigned result elsewhere in the same expression (akin to 
regex backreferences) is not a part of the basic idea actually.
It depends on the evaluation order (and whether something is evaluated 
at all), so I doubt it should even be allowed -- but even if it is, it's 
a side benefit at best.


--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 05:52:16PM +1000, Chris Angelico wrote:

> def test():
> a = 1
> b = 2
> vars = {key: locals()[key] for key in locals()}
> return vars
> 
> What would your intuition say? Should this be equivalent to dict(locals()) ?

That example is so elegant it makes me want to cry.

And not just because you shadowed the vars() builtin *wink*



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 08:30:00AM +0100, Paul Moore wrote:
> On 27 June 2018 at 07:54, Steven D'Aprano  wrote:
> > Comprehensions already run partly in the surrounding scope.
[...]
> > Given the code shown:
> >
> > def test():
> > a = 1
> > b = 2
> > result = [value for key, value in locals().items()]
> > return result
[...]

> But test() returns [1, 2]. So does that say (as you claim above) that
> "the comprehension ran in the enclosing scope"? Doesn't it just say
> that the outermost iterable runs in the enclosing scope?

I think I was careful enough to only say that this was the same result 
you would get *if* the comprehension ran in the outer scope. Not to 
specifically say it *did* run in the outer scope. (If I slipped up 
anywhere, sorry.)

I did say that the comprehension runs *partly* in the surrounding scope, 
and the example shows that the local namespace in the "... in iterable" 
part is not the same as the (sub)local namespace in the "expr for x in 
..." part.

*Parts* of the comprehension run in the surrounding scope, and parts of 
it run in an implicit sublocal scope inside a hidden function, giving us 
a quite complicated semantics for "comprehension scope":

  [expression for a in first_sequence for b in second ... ]
  |--sublocal-|local-|--sublocal--|

Try fitting *that* in the LEGB (+class) acronym :-)


This becomes quite relevant once we include assignment expressions. To 
make the point that this is not specific to := but applies equally to 
Nick's "given" syntax as well, I'm going to use his syntax:

result = [a for a in (x given x = expensive_function(), x+1, 2*x, x**3)]

Here, the assignment to x runs in the local part. I can simulate that 
right now, using locals, but only outside of a function due to CPython's 
namespace optimization inside functions. (For simplicity, I'm just going 
to replace the call to "expensive_function" with just a constant.)


py> del x
py> [a for a in (locals().__setitem__('x', 2) or x, x+1, 2*x, x**3)]
[2, 3, 4, 8]
py> x
2


This confirms that the first sequence part of the comprehension runs in 
the surrounding local scope.

So far so good. What if we move that assignment one level deep? 
Unfortunately, I can no longer use locals for this simulation, due to a 
peculiarity of the CPython function implementation. But replacing the 
call to locals() with globals() does the trick:

del x
# simulate [b*a for b in (1,) for a in (x given x = 2, x+1, 2*x, x**3)]
[b*a for b in (1,) for a in (globals().__setitem__('x', 2) or x, x+1, 2*x, 
x**3)]


That also works. But the problem comes if the user tries to assign to x 
in both the local and a sublocal section:

# no simulation here, sorry
[b*a for b in (x given x = 2, x**2) for a in (x given x = x + 1, x**3)]

That looks like it should work. You're assigning to the same x in two 
parts of the same expression. Where's the problem?

But given the "implicit function" implementation of comprehensions, I 
expect that this ought to raise an UnboundLocalError. The local scope 
part is okay:

# needs a fixed-width font for best results
[b*a for b in (x given x = 2, x**2) for a in (x given x = x + 1, x**3)]
..|-local part|.|sublocal part|

but the sublocal part defines x as a sublocal variable, shadowing the 
surrounding local x, then tries to get a value for that sublocal x 
before it is defined.

If we had assignment expressions before generator expressions and 
comprehensions, I don't think this would have been the behaviour we 
desired.

(We might, I guess, accept it as an acceptable cost of the implicit 
function implementation. But we surely wouldn't argue for this 
complicated scoping behaviour as a good thing in and of itself.)

In any case, we can work around this (at some cost of clarity and 
unobviousness) by changing the name of the variable. Not a big burden 
when the variable is a single character x:

[b*a for b in (x given x = 2, x**2) for a in (y given y = x + 1, y**3)]

but if x is a more descriptive name, that becomes more annoying. 
Nevermind, it is a way around this.

Or we could Just Make It Work by treating the entire comprehension as 
the same scope for assignment expressions. (I stress, not for the loop 
variable.) Instead of having to remember which bits of the comprehension 
run in which scope, we have a conceptually much simpler rule:

- comprehensions are expressions, and assignments inside them
  bind to the enclosing local scope, just like other expressions:

- except for the loop variables, which are intentionally
  encapsulated inside the comprehension and don't "leak".

The *implementation details* of how that works are not conceptually 
relevant. We may or may not want to advertise the fact that 
comprehensions use an implicit hidden function to do the encapsulation, 
and implicit hidden nonlocal to undo the effects of that hidden 
function. Or whatever implementation we happen to use.


> So everybody 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Paul Moore]

On 27 June 2018 at 08:52, Chris Angelico  wrote:
> On Wed, Jun 27, 2018 at 5:30 PM, Paul Moore  wrote:
>> But test() returns [1, 2]. So does that say (as you claim above) that
>> "the comprehension ran in the enclosing scope"? Doesn't it just say
>> that the outermost iterable runs in the enclosing scope?
>
> Yes - because the *outermost iterable* runs in the enclosing scope.
> But suppose you worded it like this:
>
> def test():
> a = 1
> b = 2
> vars = {key: locals()[key] for key in locals()}
> return vars
>
> What would your intuition say? Should this be equivalent to dict(locals()) ?

As I said on python-list, my intuition doesn't apply to locals() - I
simply have no idea what I'd "expect" from that code, other than a
request to go back and write it more clearly :-)

*After* staring at it for a while and trying to interpret it base on
the detailed knowledge I've gained from this thread, I'd say it does
nothing remotely useful, and if you want dict(locals()) you should
write it. (No, test() is not equivalent, because the two instances of
locals() refer to different scopes, but I can't imagine why I'd ever
need to know that outside of solving artificial puzzles like this).

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Angelico]

On Wed, Jun 27, 2018 at 5:30 PM, Paul Moore  wrote:
> But test() returns [1, 2]. So does that say (as you claim above) that
> "the comprehension ran in the enclosing scope"? Doesn't it just say
> that the outermost iterable runs in the enclosing scope?

Yes - because the *outermost iterable* runs in the enclosing scope.
But suppose you worded it like this:

def test():
a = 1
b = 2
vars = {key: locals()[key] for key in locals()}
return vars

What would your intuition say? Should this be equivalent to dict(locals()) ?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Tue, Jun 26, 2018 at 10:54:12AM +1200, Greg Ewing wrote:

> A decision still needs to be made about whether we *want*
> semantics that leak some things but not others.

My sense (or bias, if you prefer) is that the answer to that depends 
on how you word the question. If you talk about "leaking", or 
give examples with trivial 1-character names that look all too 
easy to accidentally clobber, people will say "No":

# Given this:
x = 999
[(x := i)*x for i in (1, 2)]

# should print(x) afterwards result in 4?
   

but if you show a useful example that doesn't look like an accident 
waiting to happen, but a deliberate feature:

# Given this:
previous = 0
[previous + (previous := i) for i in (1, 2, 3)]

# what value would you expect previous to have
# at the completion of the loop?

they'll be more receptive to the idea. (If they're not opposed to 
assignment expressions at all.)

Avoiding leading questions is *hard*, and I believe that in general 
people don't know what they want until they've got it. I say that from 
considering all the times I've made a radical about face, features which 
I was *sure* would be awful actually turned out to be not awful at all 
-- augmented assignment, for instance.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Paul Moore]

On 27 June 2018 at 07:54, Steven D'Aprano  wrote:
> Comprehensions already run partly in the surrounding scope.
>
> I tried to take a survey of people on the Python-List mailing list, so
> see what their expectations of comprehension scope was. Disappointingly,
> not many people responded, but those who did, invariably think in terms
> of comprehensions running inside their enclosing scope, like any other
> expression:
>
> https://mail.python.org/pipermail/python-list/2018-June/734838.html
>
> (Please excuse the doubled-up posts, some misconfigured news server is
> periodically sending duplicate posts.)
>
> (Oh and ignore my comment about Python 2 -- I was thinking of
> something else.)
>
> Given the code shown:
>
> def test():
> a = 1
> b = 2
> result = [value for key, value in locals().items()]
> return result
>
>
> nobody suggested that the result ought to be the empty list, which is
> what you should get if the comprehension ran in its own scope. Instead,
> they all expected some variation of [1, 2], which is what you would get
> if the comprehension ran in the enclosing scope.
>
> A decade or more since generator expressions started running in their
> own half-local-half-sublocal scope, people still think of scoping in
> terms of LEGB and don't think of comprehensions as running in their own
> scope *except* to the very limited degree that sometimes they are either
> surprised or pleased that "the loop variable doesn't leak".

But test() returns [1, 2]. So does that say (as you claim above) that
"the comprehension ran in the enclosing scope"? Doesn't it just say
that the outermost iterable runs in the enclosing scope?

So everybody expected the actual behaviour? (Disclaimer: in my
response, I said that I had no clear expectation, which I stand by -
locals() exposes implementation details that I don't normally feel
that I need to know - but certainly the majority of respondents
expected 1 and 2 to appear).

On the other hand,

>>> def test2():
... a = 1
... b = 2
... result = [locals().items() for v in 'a']
... return result
...
>>> test2()
[dict_items([('v', 'a'), ('.0', )])]

and I bet no-one would have expected that if you'd posed that question
(I certainly wouldn't). Although some might have said [('v', 'a')]. I
suspect some would have expected a and b to appear there too, but
that's just a guess...

So yes, it's likely that people would have found the current behaviour
unexpected in respect of locals(). But I imagine most people only care
about the effective results when referencing variables, and

>>> def test3():
... a = 1
... b = 2
... result = [a for v in (1,)]
... return result
...
>>> test3()
[1]

i.e., thanks to scope nesting, you can still reference locals from the
enclosing scope.

The problem is that := allows you to *change* values in a scope, and
at that point you need to know *which* scope. So to that extent, the
locals() question is important. However, I still suspect that most
people would answer that they would like := to assign values *as if*
they were in the enclosing scope, which is not really something that I
think people would express in answer to a question about locals().
This can be achieved with an implicit "nonlocal" (and some extra
shenanigans if the enclosing scope has a nonlocal or global
declaration itself). Which, AIUI, is what the current proposal tries
to do.

IMO, the big question over the current PEP 572 proposal is whether it
goes too far in the direction of "do what I mean". Superficially, the
semantics are pretty clearly "what people would expect", and indeed
that's been the whole focus recently to capture and satisfy *expected*
behaviour. But there are edge cases (there always are when you work
from messy real-world requirements rather than nice clean mathematical
definitions ;-)) and the question is essentially whether any of those
are bad enough to be an issue.

I'm starting to feel that they aren't, and I'm moving towards a
cautious +0 (or even +1) on the proposal.
Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Tue, Jun 26, 2018 at 05:42:43AM +1000, Chris Angelico wrote:

> So. sublocal scopes, like in the earliest versions of PEP 572?
> 
> The wheel turns round and round, and the same spokes come up.

It isn't as if comprehensions (and generator expressions) run in a 
proper separate scope. It is more half-and-half, sometimes it is 
seperate, sometimes it isn't:


py> def show_subscope():
... a, b = 1, 2
... print("Comprehension scope, Part A")
... print(next(locals() for x in (1,)))
... print("Comprehension scope, Part B")
... print(next(obj for obj in (locals(),)))
...
py> show_subscope()
Comprehension scope, Part A
{'x': 1, '.0': }
Comprehension scope, Part B
{'b': 2, 'a': 1}


Comprehensions already run partly in the surrounding scope.

I tried to take a survey of people on the Python-List mailing list, so 
see what their expectations of comprehension scope was. Disappointingly, 
not many people responded, but those who did, invariably think in terms 
of comprehensions running inside their enclosing scope, like any other 
expression:

https://mail.python.org/pipermail/python-list/2018-June/734838.html

(Please excuse the doubled-up posts, some misconfigured news server is 
periodically sending duplicate posts.)

(Oh and ignore my comment about Python 2 -- I was thinking of 
something else.)

Given the code shown:

def test():
a = 1
b = 2
result = [value for key, value in locals().items()]
return result


nobody suggested that the result ought to be the empty list, which is 
what you should get if the comprehension ran in its own scope. Instead, 
they all expected some variation of [1, 2], which is what you would get 
if the comprehension ran in the enclosing scope.

A decade or more since generator expressions started running in their 
own half-local-half-sublocal scope, people still think of scoping in 
terms of LEGB and don't think of comprehensions as running in their own 
scope *except* to the very limited degree that sometimes they are either 
surprised or pleased that "the loop variable doesn't leak".

For example:

http://nbviewer.jupyter.org/github/rasbt/python_reference/blob/master/tutorials/scope_resolution_legb_rule.ipynb

doesn't mention comprehensions until the very end, almost in passing, 
and doesn't describe them as a separate scope at all. Rather, they are 
described as using closures "to prevent the for-loop variable to cut 
[sic] into the global namespace."

This doesn't mention comprehension subscope at all:

https://www.python-course.eu/python3_global_vs_local_variables.php

Even the official documentation doesn't explicitly state that 
comprehensions are a separate scope:

https://docs.python.org/3/reference/executionmodel.html#resolution-of-names

rather leaving it to an after thought, to mention in passing almost as 
if it were an implementation-dependent accident, that comprehensions 
cannot see variables defined in any surrounding class scope.

Aside from the loop variable (which PEP 572 will not change!) I see no 
evidence that the average non-core developer Python programmer considers 
comprehensions as a separate scope, or wants them to be a separate 
scope. Regardless of comprehensions being implicitly wrapped in a 
function or not, the average developer doesn't want the loop variable to 
"leak", and that's as far as their consideration has needed to go until 
now. But when pressed to explicitly consider the scope inside a 
comprehension, the evidence I have seen is that they consider it the 
same as the local scope surrounding it.

Which is not wrong, as can be seen from the example above.

Unlike the loop variable, I don't believe that assignment-expression 
bindings quote-unquote "leaking" from comprehensions will come as a 
surprise. On the contrary -- given that Nick et al have gone to great 
lengths to ensure that as a first approximation, comprehensions are 
equivalent to a simple for-loop running in the current scope:

result = [expr for a in seq]

# is almost the same as
result = []
for a in seq:
result.append(expr)

I expect that people will be surprised if explicit, non-loop 
variable assignments *don't* occur in the current scope.

If all that takes to implement is something like an implicit "nonlocal", 
that's hardly worse than the implicit functions already used.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com