Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Guido van Rossum]

On Tue, Jul 3, 2018 at 4:25 PM Victor Stinner  wrote:

> 2018-07-02 20:19 GMT+02:00 Guido van Rossum :
> > Thank you all. I will accept the PEP as is. (...)
>
> I see more and more articles ("on the Internet") saying that Guido van
> Rossum already accepted the PEP. Is the PEP already accepted or will
> be accepted?
>
> Right now, https://www.python.org/dev/peps/pep-0572/ status is "Draft".
>

That's a rather philosophical question. I clearly said "I will" not "I
might". And if you're asking whether it's likely that I'll change my mind,
no. I would like help with updates to the PEP to summarize some of the
discussions and rejected proposals. And I am giving myself a week to "cool
off". But I am muting threads that bring up objections that I've heard
before (e.g. "design principles"). So those articles aren't wrong. Your
patience is appreciated.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Victor Stinner]

2018-07-02 20:19 GMT+02:00 Guido van Rossum :
> Thank you all. I will accept the PEP as is. (...)

I see more and more articles ("on the Internet") saying that Guido van
Rossum already accepted the PEP. Is the PEP already accepted or will
be accepted?

Right now, https://www.python.org/dev/peps/pep-0572/ status is "Draft".

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Terry Reedy wrote:
If we had followed the math precedent, instead of language>, we would have set builders, list builders, dict builders, and 
generator builders.


I was intending to suggest something like that back when
comprehensions were first being discussed, but people
raced ahead and adopted the term "comprehension" before
I got the chance.

"List builder" and "dict builder" make a lot of sense,
but "generator builder" not so much -- it *is* a generator,
not something that builds a generator. In fact it doesn't
build anything in the sense that the others do. So maybe
"generator expression" is the best we could have done.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Guido van Rossum]

I smell a new thread. :-)

On Tue, Jul 3, 2018 at 3:06 PM Terry Reedy  wrote:

> On 7/3/2018 2:42 AM, Tim Peters wrote:
>
> > So if we had it to do over again I'd sigh and accept "generator
> > comprehensions" anyway.  It's been an eternal PITA - and especially in
> > the PEP 572 threads! - to keep typing "comprehensions or generator
> > expressions".  Then again, if I had the power of Guido's time machine,
> > I'd go back more, and not use "comprehensions" for anything to begin
> > with.
>
> Amen.  I cannot make 'comprehension' in this context comprehensible
> without some linguistic twisting.
>
> >  Instead we'd have list, dict, set, and generator twizzlers,
> > affectionately called listwiz, dictwiz, setwiz, and gentwiz by the cool
> > kids :-)
>
> I learned the set notion, such as
> {n^2: n in  N; 1 <= n < 100, n even}  # math
> {n*n for n in range(1,100) if not n%2}  # python
> as 'set builder' notation.
>
> If we had followed the math precedent, instead of  language>, we would have set builders, list builders, dict builders, and
> generator builders.
>
> I half seriously think we should consider this for 3.8 for the benefit
> of future Python programmers as well as ourselves.  Comprehensions that
> can contain assignment expressions are a slightly new thing.
>
>
> --
> Terry Jan Reedy
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Terry Reedy]

On 7/3/2018 2:42 AM, Tim Peters wrote:

So if we had it to do over again I'd sigh and accept "generator 
comprehensions" anyway.  It's been an eternal PITA - and especially in 
the PEP 572 threads! - to keep typing "comprehensions or generator 
expressions".  Then again, if I had the power of Guido's time machine, 
I'd go back more, and not use "comprehensions" for anything to begin 
with.


Amen.  I cannot make 'comprehension' in this context comprehensible 
without some linguistic twisting.


  Instead we'd have list, dict, set, and generator twizzlers, 
affectionately called listwiz, dictwiz, setwiz, and gentwiz by the cool 
kids :-)


I learned the set notion, such as
{n^2: n in  N; 1 <= n < 100, n even}  # math
{n*n for n in range(1,100) if not n%2}  # python
as 'set builder' notation.

If we had followed the math precedent, instead of language>, we would have set builders, list builders, dict builders, and 
generator builders.


I half seriously think we should consider this for 3.8 for the benefit 
of future Python programmers as well as ourselves.  Comprehensions that 
can contain assignment expressions are a slightly new thing.



--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Mon, Jul 2, 2018 at 11:42 PM, Tim Peters  wrote:


> "comprehensions" was always a dubious term, carried over from set theory
> where the term focuses on the optional "if" part rather than the more
> fundamental iterator or computation parts.
>

I always wondered about that :-) -- I'd say for most of us that aren't
familiar with set theory, it's kind of a "sounds something like putting
thing together" word and I just left it at that, and learned what they are.


> So if we had it to do over again I'd sigh and accept "generator
> comprehensions" anyway.  It's been an eternal PITA - and especially in the
> PEP 572 threads! - to keep typing "comprehensions or generator
> expressions".
>

Well, too late to change the official name, but not too late to start using
the term in threads like these -- and other documentation, etc

I find there is a lot of confusion about the word "generator", as it
implies a "thing that generates values on the fly" (like, say the range()
object.

But then, in Python, a generator is something that gets crated by a
generator function, and CAN be an "thing (iterator) that generates things
on the fly", but can also be a more generic coroutine, and can be used in
nifty ways that really have nothing to do with generating a bunch of value.
(like pytest fixtures, for example)

So we have generators, iterators, and iterables, and generators can be
iterators, but aren't always, and any number of iterators can generate
values on the fly, and 

so it's all a bit of a mess to explain to a newbie.

Then again, if I had the power of Guido's time machine, I'd go back more,
> and not use "comprehensions" for anything to begin with.  Instead we'd have
> list, dict, set, and generator twizzlers, affectionately called listwiz,
> dictwiz, setwiz, and gentwiz by the cool kids :-)
>

I'd like that!

-CHB




-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Rob Cliffe via Python-Dev]

On 03/07/2018 06:47, Tim Peters wrote:

[Rob Cliffe]
> It's late to raise this,

By months, yes ;-)

> but what exactly are the objections to the syntax
>         expr -> name  # or variations such as  expr => name
> instead of
>         name := expr

>
> The PEP mentions that this syntax does not have a problem that "as"
> does, but does not list any downsides of it.

My guess:  it probably strikes too many as "excessive novelty",  
 These are assignment expressions.  Python's assignment statements put 
the target at the left.  Why change that?  ":=" is used for assignment 
in many more other languages than "->" is.  Why fight that?


> It conforms to left-to-right evaluation, where name:=expr does not.

?  Only "expr" is evaluated, so left-to-right seems irrelevant here.  
The "evaluation" of a simple name as a binding target is a no-op (no 
code is generated).  If you really do see this as a wart anyway, then 
it's positively a Good Thing that it's exactly the same "wart" as in 
Python's assignment statements.


> It (I would argue) reduces the asymmetry of the first use of a
> sub-expression in cases such as
>     [ ( (f(x) -> y)**2, y**3, y**4) for x in iterable ]
>  vs
>     [ ( (y := f(x))**2, y**3, y**4) for x in iterable ]
> because the first "y" is closer to the way it is used, viz "**2".

The first form reads a little better to me too, but not a lot better.  
The problem I have with variations of this example on its own (which 
comes up surprisingly often with minor changes) is that it's clearer 
spelled today via


    [(y**2, y**3, y**4) for y in map(f, iterable)]

Spelling that with either form of assignment expression reads 
significantly worse than that to my eyes


But more importantly, it's expected that assignment expressions will 
be used _most_ often to make some common `if` and `while` patterns 
briefer.  Hardly all. Our eyes are already trained to "look at the far 
right end" for the value being tested, and, e.g.,


    while data := sock.recv():

preserves that.  Especially in code that doesn't _always_ use 
assignment expressions in such contexts (which is likely all 
significant blobs of code), it would be visually jarring to have to 
"sometimes look in the middle instead" to extract the important part of:


    while sockrecv() -> data:

"Look to the left for the name, look to the right for the value" is 
the rule for assignment statements, assignment expressions, and `for` 
loop targets.


But there's no "QED" here because this isn't a deductive science.  The 
final answer is "because that's what Guido liked best" ;-)

Thanks, Tim, for a thoughtful answer.
Don't get me wrong, I feel quite happy with ":=".  Perhaps you have 
managed to articulate some thoughts that were buried in my 
subconscious.  It's just that I couldn't come up with any rational 
objections to "->".

Rob Cliffe



 
	Virus-free. www.avg.com 
 



<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Chris Barker]

> > However, generator expressions ( why don’t we call them generator

> > comprehensions?)

Because nobody really liked the "iterator comprehensions" or "accumulator
displays" they were variously called at the start.

https://mail.python.org/pipermail/python-dev/2003-October/039186.html

As that explains, "generator expressions" was an attempt to break away from
that "comprehensions" was always a dubious term, carried over from set
theory where the term focuses on the optional "if" part rather than the
more fundamental iterator or computation parts.

At the start, for some (forgotten by me) reason it seemed important to make
a distinction between "things like this" that were evaluated at once (list,
dict, and set comprehensions) and the new-fangled accumulator displays that
got evaluated lazily.  But the "generator" in "generator comprehensions"
would really be enough all by itself to make that clear enough.

So if we had it to do over again I'd sigh and accept "generator
comprehensions" anyway.  It's been an eternal PITA - and especially in the
PEP 572 threads! - to keep typing "comprehensions or generator
expressions".  Then again, if I had the power of Guido's time machine, I'd
go back more, and not use "comprehensions" for anything to begin with.
Instead we'd have list, dict, set, and generator twizzlers, affectionately
called listwiz, dictwiz, setwiz, and gentwiz by the cool kids :-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Rob Cliffe]
> It's late to raise this,

By months, yes ;-)

> but what exactly are the objections to the syntax

> > expr -> name  # or variations such as  expr => name

> > instead of

> > name := expr

> >

> > The PEP mentions that this syntax does not have a problem that "as"

> > does, but does not list any downsides of it.

My guess:  it probably strikes too many as "excessive novelty",   These are
assignment expressions.  Python's assignment statements put the target at
the left.  Why change that?  ":=" is used for assignment in many more other
languages than "->" is.  Why fight that?

> It conforms to left-to-right evaluation, where name:=expr does not.

?  Only "expr" is evaluated, so left-to-right seems irrelevant here.  The
"evaluation" of a simple name as a binding target is a no-op (no code is
generated).  If you really do see this as a wart anyway, then it's
positively a Good Thing that it's exactly the same "wart" as in Python's
assignment statements.

> It (I would argue) reduces the asymmetry of the first use of a
> sub-expression in cases such as

> > [ ( (f(x) -> y)**2, y**3, y**4) for x in iterable ]

> >  vs

> > [ ( (y := f(x))**2, y**3, y**4) for x in iterable ]

> > because the first "y" is closer to the way it is used, viz "**2".

The first form reads a little better to me too, but not a lot better.  The
problem I have with variations of this example on its own (which comes up
surprisingly often with minor changes) is that it's clearer spelled today
via

[(y**2, y**3, y**4) for y in map(f, iterable)]

Spelling that with either form of assignment expression reads significantly
worse than that to my eyes

But more importantly, it's expected that assignment expressions will be
used _most_ often to make some common `if` and `while` patterns briefer.
Hardly all.  Our eyes are already trained to "look at the far right end"
for the value being tested, and, e.g.,

while data := sock.recv():

preserves that.  Especially in code that doesn't _always_ use assignment
expressions in such contexts (which is likely all significant blobs of
code), it would be visually jarring to have to "sometimes look in the
middle instead" to extract the important part of:

while sock.recv() -> data:

"Look to the left for the name, look to the right for the value" is the
rule for assignment statements, assignment expressions, and `for` loop
targets.

But there's no "QED" here because this isn't a deductive science.  The
final answer is "because that's what Guido liked best" ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker - NOAA Federal via Python-Dev]

> On Jul 2, 2018, at 8:34 AM, Steven D'Aprano  wrote:

Guido has decided — and despite my concerns, I’m going to enjoy my new
loop-and-a half construct:-)

But a comment on this:

> comprehension are no more special than
> assignments inside any other expression. They bind in the current scope,
> same as always, and keep the sensible identity that these two
> expressions are exactly equivalent in their visible semantics:
>
>   [x:=0, x:=1, x:=2]
>
>   [x:=i for i in (0, 1, 2)]
>
> including assignments.

Sure — and I don’t think that’s confusing.

However, generator expressions ( why don’t we call them generator
comprehensions?) are a different story, as they may be run at some
arbitrary time in the future. This hasn’t been an issue (except for
the loop variable, which has been addressed) because:

1) Much of the time, the gen_ex is run right away, in-line.

2) There aren’t many ways to manipulate the local namespace in a gen_ex.

With assignment expressions, it will be much easier to manipulate the
local namespace, so there is room for some real confusion here.

So a real local namespace gen_exp (and comprehensions, for
consistency) would be nice.

However, that ship has pretty much sailed.

Will it end up being a common problem? Probably not, because (a) is
still the case, and := will be used infrequently, and hopefully with
unlikely to clash names.

And as for all the other languages that have assignment expressions?
Do they have constructs like generator expressions?

-CHB
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Rob Cliffe via Python-Dev]

On 02/07/2018 19:19, Guido van Rossum wrote:
Thank you all. I will accept the PEP as is. I am happy to accept 
*clarification* updates to the PEP if people care to submit them as 
PRs to the peps repo (https://github.com/python/peps), and that could 
even (to some extent) include summaries of discussion we've had, or 
outright rejected ideas. But even without any of those I think the PEP 
is very clear so I will not wait very long (maybe a week).



It's late to raise this, but what exactly are the objections to the syntax
        expr -> name  # or variations such as  expr => name
instead of
        name := expr

The PEP mentions that this syntax does not have a problem that "as" 
does, but does not list any downsides of it.

It conforms to left-to-right evaluation, where name:=expr does not.
It (I would argue) reduces the asymmetry of the first use of a 
sub-expression in cases such as

    [ ( (f(x) -> y)**2, y**3, y**4) for x in iterable ]
vs
    [ ( (y := f(x))**2, y**3, y**4) for x in iterable ]
because the first "y" is closer to the way it is used, viz "**2".

Regards
Rob Cliffe
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Michael Selik]
>>> My worry is that assignment expressions will add about 15 to 20
>>> minutes to my class and a slight discomfort.

[Tim]
>> So not intractable - which is my high-order bit ;-)
>>
>> For those who want more bits of precision (perhaps Guido), while
>> quantification is good, it needs context to provide insight.  Like,
>> out of how many class hours total?

[Michael]
> Generally between 20 and 40 hours.

Ah - so I take it you're not teaching raw computer beginners, but people
who already know how to program in some other language(s)?  If so, perhaps
you could leverage on that assignment expressions are already present in
the most heavily used languages.  Depends on the students' backgrounds, of
course.

>> Is 15-20 minutes a little, a lot, par for the course ... compared to
other topics?

> I guessed 15-20 minutes, because I'm mentally comparing it to things
> like ternary expressions. Odds and ends that make the code better,
> but not a major concept that deserves hours.

That's where I see it fitting too.  While I don't expect it will ever come
up, if I were tasked with teaching assignment expressions, I can picture
ways of doing it that would take anywhere from one minute to two hours,
depending on the target audience's backgrounds, interests, and needs ("two
hours" for those fascinated by computer language history).


>> Will it require you to drop other topics?

> Yes. It might not seem like much,

Nope!  It was a trick question.  If you had answered "no", I would have
known you were just making things up ;-)

> but every minute counts. I'd probably try to ignore := unless some
> pesky student brings it up. It's like someone saying, "Hey, I heard
> that Python can't do threads?!" I always say, "Good question," but
> internally I'm thinking, "there goes a half hour. What can I cut today?"

Absolutely.  40 hours can't possibly cover more than a significant overview
of high-order bits.


>> Would you _save_ twice as much class time if we got rid of "is"? ;-)

> Ha. You joke, but ``is`` takes about 5 minutes. About 5 or 10 minutes
> more if some clever student notices that ``1 is 1`` and I need to explain
> Singletons and interpreter optimizations versus language spec.

That surprised me!  Educators have often said "is" was hard to teach, and
it's one of the F'est of Python FAQs on StackOverflow.  I always figured
that's because it's trivial if you have deep understanding of Python's
conceptual object model, but appears to be a random Boolean generator if
you're just mucking around at a shell without that understanding.  Still,
something like this sometimes temporarily baffles even experts:

>>> [] is []  # OK, `[]` always creates a new list
False
>>> id([]) == id([])  # or does it???
True

>> If it's accepted, do read the PEP

> I've read it a few times now. I hope I didn't sound like I haven't
> read it. That'd be embarrassing.

Heh.  No, I just wanted to be sure.  It's just impossible to tell from a
discussion that's entirely "meta".

> ...
> From my brief observations, it seems that the nattering nabobs of
negativism,
> such as myself, are mostly educators.

In this specific thread, sure, but I expect that's because it has "educator
feedback" in the Subject.  There's been plenty of opposition from
non-educators in other threads.  I like this thread because it's been more
civil than most :-)

> I recently started to wonder if I'd care so much about the language if I
> didn't teach. I suspect that if I didn't worry about teaching new
features,
> Python 4 could be announced tomorrow and I wouldn't really mind.

Sure - and I wouldn't care so much (or, indeed, at all) if Python wasn't my
language of choice for most projects for over 20 years now.

> I suppose it is selfish. But I hope that you [Tim], Guido, and the so
> many others who have poured energy into this project will appreciate
> that it's not the current users, but the next billion (?!) Pythonistas
that
> will really keep the language going.

Which, perhaps paradoxically, is why I'm generally in favor of even small
(but general) improvements. regardless of short-term costs:  so the next
billion Pythonistas can benefit.

> Maintaining popularity among educators is a big part of that.

Oh, I have no idea about that.  I'm at a loss for what accounts for
longer-term language popularity, so I push for things that appeal to me as
a programmer with broad and deep experiences.  Everyone who cares about the
language _should_ be heard, but their motivations don't even need to
overlap.

When I started college, I took classes in every language for which there
was a class: assembler, FORTRAN, LISP, and SNOBOL4.  Three of those survive
as niche languages now, and the last is long dead (damned shame, too!
SNOBOL4 was brilliantly creative).

The last time I had significant contact with academia, Pascal was all the
rage (yes, that dates me).  Not only was it universally loved by educators,
it was in exactly the right place at exactly the right 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Guido van Rossum]

Thank you all. I will accept the PEP as is. I am happy to accept
*clarification* updates to the PEP if people care to submit them as PRs to
the peps repo (https://github.com/python/peps), and that could even (to
some extent) include summaries of discussion we've had, or outright
rejected ideas. But even without any of those I think the PEP is very clear
so I will not wait very long (maybe a week).

On Mon, Jul 2, 2018 at 8:38 AM Steven D'Aprano  wrote:

> On Wed, Jun 27, 2018 at 07:29:52PM -0500, Tim Peters wrote:
> [...]
> > For example, if the name is declared "global" in the outer scope, you'll
> > get a compile-time error if you try to declare it "nonlocal" in the
> > contained scope.  "parentlocal" adjusts its meaning accordingly,
> becoming a
> > synonym for "global" in that specific case.
>
> "Parentlocal" is only a thing if we buy into the paradigm that inside
> comprehensions is a separate "local". And *that* is only true under
> two circumstances:
>
> - if you are utterly immersed in the implementation of comprehensions
>   as invisible, implicit functions;
>
> - or if you start from the premise that comprehensions ought to
>   encapsulate not just the loop variable, but anything else as well.
>
>
> But experimenting with locals() inside comprehensions shows that
> comprehension-scope *isn't* a well-defined thing. It already bleeds out
> of the comprehension, and so would some (but only some!) assignment
> expressions.
>
> Instead, if we start from the premise that comprehensions (like any
> other expression) run in the current scope, then there is no need to
> invent a term "parentlocal". There's just the usual LEGB scopes, plus
> class (which people usually forget).
>
> With no sublocal scopes (a term we never even had prior to this PEP)
> assignments inside the comprehension are no more special than
> assignments inside any other expression. They bind in the current scope,
> same as always, and keep the sensible identity that these two
> expressions are exactly equivalent in their visible semantics:
>
> [x:=0, x:=1, x:=2]
>
> [x:=i for i in (0, 1, 2)]
>
> including assignments.
>
> What about the loop variable?
>
> They ARE special, which is completely justified by the Zen:
>
> Although practicality beats purity.
>
> We can take a series of ever-more-detailed explanations, starting from
> the highest "bird's eye" view and gradually dropping further into the
> murky details of the implementation when, and if, required:
>
> - assignment within comprehensions is no different from assignment
>   in any other expression, it occurs in the local scope;
>
> - loop variables? they're a special case, for good reason, and are
>   encapsulated inside the comprehension;
>
> - how? they're hidden in an implicit, invisible scope, same as .0
>   the implicit, invisible iterator object;
>
> - oh, you didn't know about the .0 variable? well forget about it,
>   it's an undocumented implementation detail, just like the invisible,
>   implicit function used by comprehensions;
>
> - oh, you didn't know about that either? read the source code.
>
>
> Only the first two levels of explanation are part of Python the
> language. The rest is CPython implementation.
>
>
>
> --
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 07:29:52PM -0500, Tim Peters wrote:
[...]
> For example, if the name is declared "global" in the outer scope, you'll
> get a compile-time error if you try to declare it "nonlocal" in the
> contained scope.  "parentlocal" adjusts its meaning accordingly, becoming a
> synonym for "global" in that specific case.

"Parentlocal" is only a thing if we buy into the paradigm that inside 
comprehensions is a separate "local". And *that* is only true under 
two circumstances:

- if you are utterly immersed in the implementation of comprehensions
  as invisible, implicit functions;

- or if you start from the premise that comprehensions ought to
  encapsulate not just the loop variable, but anything else as well.


But experimenting with locals() inside comprehensions shows that 
comprehension-scope *isn't* a well-defined thing. It already bleeds out 
of the comprehension, and so would some (but only some!) assignment 
expressions.

Instead, if we start from the premise that comprehensions (like any 
other expression) run in the current scope, then there is no need to 
invent a term "parentlocal". There's just the usual LEGB scopes, plus 
class (which people usually forget).

With no sublocal scopes (a term we never even had prior to this PEP) 
assignments inside the comprehension are no more special than 
assignments inside any other expression. They bind in the current scope, 
same as always, and keep the sensible identity that these two 
expressions are exactly equivalent in their visible semantics:

[x:=0, x:=1, x:=2]

[x:=i for i in (0, 1, 2)]

including assignments.

What about the loop variable?

They ARE special, which is completely justified by the Zen:

Although practicality beats purity.

We can take a series of ever-more-detailed explanations, starting from 
the highest "bird's eye" view and gradually dropping further into the 
murky details of the implementation when, and if, required:

- assignment within comprehensions is no different from assignment
  in any other expression, it occurs in the local scope;

- loop variables? they're a special case, for good reason, and are
  encapsulated inside the comprehension;

- how? they're hidden in an implicit, invisible scope, same as .0 
  the implicit, invisible iterator object;

- oh, you didn't know about the .0 variable? well forget about it,
  it's an undocumented implementation detail, just like the invisible,
  implicit function used by comprehensions;

- oh, you didn't know about that either? read the source code.


Only the first two levels of explanation are part of Python the 
language. The rest is CPython implementation.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 03:31:42PM -0700, Guido van Rossum wrote:

> I'd also like to keep the rule prohibiting use of the same name as a
> comprehension loop control variable and as an inline assignment target;
> this rule would also prohibit shenanigans with nested comprehensions (for
> any set of nested comprehensions, any name that's a loop control variable
> in any of them cannot be an inline assignment target in any of them). This
> would also apply to the "outermost iterable".

+1


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Matt Arcidy]

On Mon, Jul 2, 2018 at 2:34 AM Michael Selik  wrote:
>
> On Sun, Jul 1, 2018 at 8:21 PM Matt Arcidy  wrote:
>>
>> [...] Can anyone adequately explain why this specific modality of learning,  
>> a student-in-a-seat based educator, must outweigh all other modalities [...]?
>
>
> 1. It doesn't.
> 2. It's a proxy for the other modes.
>
> I hope this was an adequate explanation.

Absolutely, thank you.  We agree it doesn't out weigh other methods.
Clearly I disagree about the proxying.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Michael Selik]

On Sun, Jul 1, 2018 at 8:21 PM Matt Arcidy  wrote:

> [...] Can anyone adequately explain why this specific modality of
> learning,  a student-in-a-seat based educator, must outweigh all other
> modalities [...]?


1. It doesn't.
2. It's a proxy for the other modes.

I hope this was an adequate explanation.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Michael Selik]

On Sun, Jul 1, 2018 at 11:36 PM Tim Peters  wrote:

> [Michael Selik]
> > My worry is that assignment expressions will add about 15 to 20
> > minutes to my class and a slight discomfort.
>
> So not intractable - which is my high-order bit ;-)
>
> For those who want more bits of precision (perhaps Guido), while
> quantification is good, it needs context to provide insight.  Like, out of
> how many class hours total?
>

Generally between 20 and 40 hours.


Is 15-20 minutes a little, a lot, par for the course ... compared to other
> topics?
>

I guessed 15-20 minutes, because I'm mentally comparing it to things like
ternary expressions. Odds and ends that make the code better, but not a
major concept that deserves hours.


Will it require you to drop other topics?
>

Yes. It might not seem like much, but every minute counts. I'd probably try
to ignore := unless some pesky student brings it up. It's like someone
saying, "Hey, I heard that Python can't do threads?!" I always say, "Good
question," but internally I'm thinking, "there goes a half hour. What can I
cut today?"



> Would you _save_ twice as much class time if we got rid of "is"? ;-)
>

Ha. You joke, but ``is`` takes about 5 minutes. About 5 or 10 minutes more
if some clever student notices that ``1 is 1`` and I need to explain
Singletons and interpreter optimizations versus language spec.


If it's accepted, do read the PEP
>

I've read it a few times now. I hope I didn't sound like I haven't read it.
That'd be embarrassing.


Meta: About the Vasa, I'm not concerned.
>

Matt Arcidy brought up an interesting point, which I'll quote here: "... I
don't see any importance to the position of educators right now, especially
since these educators in the thread are complaining about an increase in
their personal work, for which it appears they were compensated."

>From my brief observations, it seems that the nattering nabobs of
negativism, such as myself, are mostly educators. I recently started to
wonder if I'd care so much about the language if I didn't teach. I suspect
that if I didn't worry about teaching new features, Python 4 could be
announced tomorrow and I wouldn't really mind.

I suppose it is selfish. But I hope that you [Tim], Guido, and the so many
others who have poured energy into this project will appreciate that it's
not the current users, but the next billion (?!) Pythonistas that will
really keep the language going. Maintaining popularity among educators is a
big part of that.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Antoine Pitrou]

On Mon, 2 Jul 2018 10:25:42 +1000
Steven D'Aprano  wrote:
> 
> How do people who teach other languages deal with this?
> 
> Assignment expressions are hardly a new-fangled innovation of Python's. 
> They're used in Java, Javascript, Ruby, Julia, R, PHP and of course 
> pretty much the entire C family (C, C++, C# at least).

Those other languages don't have two different assignment operators,
AFAIK.  That's the main point of complication PEP 572 introduces, not
the fact that assignment can now be used in an expression.

> Admittedly R has the advantage that they don't have to teach a distinct 
> assignment syntax and explain *why* it ought to be distinct. But 
> countering that, they have *four* different ways of doing assignment.

I don't think R is easy to understand. It depends on the demographics.
For a software engineer like me, it's pretty hard to wrap my head around
R's nonsense.  I think R is only easy if you accept to use it in a
"tinker aimlessly until my code works" manner.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Antoine Pitrou]

Is this message some kind of joke or did you just send it to the wrong
mailing-list/recipient?


On Sun, 1 Jul 2018 20:21:19 -0700
Matt Arcidy  wrote:
> This cynical view on students is shocking!  Everyone on this list has
> been a student or a learner for far longer than an educator, and the
> perspective from students and learners are far more important than
> educators to assess this angle regardless.  Can anyone adequately
> explain why this specific modality of learning,  a student-in-a-seat
> based educator, must outweigh all other modalities learners use to
> increase knowledge and skill, from the perspectives of policy, tool
> creation, and each of our time spent learning?
> 
> Shortest story:
> Teach not to re-use names.
> 
> Short story:
> 1) What about the full mosaic of learning vs. this myopic view on
> seat-based student-educator interaction?
> 2) What about smart, motivated, diligent and cautious students?
> 3) What weight should educator opinion be given with respect to
> providing convenience to professional Python programmers?
> 4) Who is this Student Stupid von Densemeister anyways?
> 5) Are assignment expressions convenience and is any danger the pose
> unmitagatble?
> 6) Consider adding an "Important Topics not Covered" or "Further
> Reading" reading section to your class description
> 7) Creating examples showing this effect is easy, especially when not
> actually re-using the name in the expression for explanatory purposes.
> it's the same as creating examples showing how re-use works in
> comprehensions.
> 
> 
> Let's stop constructing these fake Students.  They only work as
> appeals to the people we have come across whose lack of understanding
> has made our life painful.  This construction is actively filtering
> all the good students for the sake of influencing this decision, yet
> again punishing or discounting the intelligent, quick, and diligent.
> 
> And what of this underlying premise that educator's should
> _significantly_ influence language development?  Limiting Python's
> tools to Student Straw-man's ability to learn is just dissonant, they
> have nothing to do with each other, nor does this cause-effect
> relationship actually exist.   Let's evaluate this reductionist
> statement:
> "I understand X, but this other person is not capable of understanding
> X, therefore X should not exist"  Is has there ever been an X for
> which this is true, let alone the backwardation necessary to fully
> close the statement?
> 
> The actual argument is far less reductionist, yet even more ridiculous:
> "I understand X,  this other person may take time to learn X, and may
> use X wrong, therefore X should not exist"
> "I understand assignment expressions, but this other class of person
> may take time to learn assignment expressions, and may use assignment
> expressions wrong, therefore assignment expressions should not be
> accepted"
> 
> Rhetorically I disagree with how teaching is being presented, to the
> point of near insult (for me lacking a better term).  You are saying
> these statements about _my_ learning path, (though not personally of
> course.)  Each of you occupied a role of student at some point, and
> each of these statements are being made about your path as well.  Do
> these ring true of your student experience?  What about your much
> broader experience as a _learner_?  You think a tool shouldn't exist
> because it took you time to learn it and you wrote some hard to debug
> code, and possibly crashed production, got fired, lost your house and
> your pet snake, and crashed the planet into the sun?
> 
> Now I yield, I will accept this position: all/some students cannot
> learn this (or it's too complex to teach), but they must learn this
> during some class to quickly become effective python developers.  How
> much weight should this position have in this decision?  Let's appeal
> to the learner in us.  How much of our learner's path, percentage of
> total time learning all things python related, has been in a seat
> listening to someone else, and that's the only place from which we
> gained the knowledge to meet the educator's objective?  This time
> spent in a class, how does that compare to hours in other learning
> modalities?  Is this percentage not exactly the weight assigned to
> that position?  Are people hired from pure class-room based experience
> expected to require zero further learning?  Are people more valuable
> based on classroom hours or work hours?
> 
> As for handling teaching the subject or not, this is easily remedied
> with how I do it: "Important Topics not Covered", with resources.
> 
> Anyone here can rightfully claim educator status by having taught
> another person something related to this language, which includes
> at-work mentoring, informal discussions, posting/replying on SO,
> blogging, etc.  Are they not being solicited to comment as well?  It's
> possible to answer this question while vehemently disagreeing with the
> PEP.  This 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Tim]
>> ...
>> So, ya, when someone claims [assignment expressions will] make
>> Python significantly harder to teach, I'm skeptical of that claim.

[Michael Selik]
> I don't believe anyone is making that claim.

I haven't seen it in this specific thread, but the larger discussion has
been going on for several months.

> My worry is that assignment expressions will add about 15 to 20
> minutes to my class and a slight discomfort.

So not intractable - which is my high-order bit ;-)

For those who want more bits of precision (perhaps Guido), while
quantification is good, it needs context to provide insight.  Like, out of
how many class hours total?  Is 15-20 minutes a little, a lot, par for the
course ... compared to other topics?  Will it require you to drop other
topics?  Would you _save_ twice as much class time if we got rid of "is"?
;-)


> As Mark and Chris said (quoting Mark below), this is just one straw in
> the struggle against piling too many things on the haystack. Unlike
> some changes to the language, this change of such general use that
> it won't be an optional topic. Once widely used, it ain't optional.

If it's accepted, do read the PEP - while the syntax will _allow_
assignment expressions just about everywhere, the recommended examples are
all simple & straightforward.  "If it's not obviously better, don't use it"
is excellent advice.

Because it's allowed almost everywhere, I expect that unanticipated "good
uses" will pop up, but at the start a high majority of good uses seem to
fall into a small number of patterns.

Meta:   About the Vasa, I'm not concerned.  C++ has around 150 people
relentlessly writing an endless sequence of enhancement proposals largely
aimed at highly esoteric expert applications of an already extraordinarily
complex language.  Bjarne Stroustrup is right to be concerned about that.
His goal is to cut back on the complications striving for theoretical
perfection in all conceivable applications, and go back to working on ideas:

that can be used by “ordinary programmers” whose main concern is to ship
> great applications on time

http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p0977r0.pdf

If C++ hadn't inherited assignment expressions from C from the start(*), I
expect that's an idea he'd _want_ to consider now.  They're well within the
grasp of "ordinary programmers" - if they can master `j = 3`, they're 90%
of the way to mastering `j := 3` (although it may be that "good taste"
can't be taught at all).

(*) Yes, I know about the stuff added to support yet another form of
assignment expression in `if` and `switch` headers in C++ 17.  That
appeared to be more for "theoretical purity".  Assignment expressions were
always allowed there, but previously only `for` headers allowed _declaring_
a new variable inside the parens.  Now `if` and `switch` allow that too.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Sun, Jul 1, 2018 at 5:25 PM, Steven D'Aprano  wrote:

>
> > an optional topic. Once widely used, it ain't optional.
>
> Without knowing the details of your course, and who they are aimed at,
> we cannot possibly judge this comment. Decorators are widely used, but
> surely you don't teach them in a one day introductory class aimed at
> beginners?
>
> Here is the syllabus for a ten week course:
>
> https://canvas.uw.edu/courses/1026775/pages/python-100-course-syllabus
>

That would be mine ;-)


> Note that decorators and even regular expressions don't get touched
> until week ten.


I actually often don't ever get to regex -- why? because they are a topic
all of their own, and not unique to Python. And if code uses a regex, it
doesn't change anything about any other code anywhere. IN sort, regex's are
a library, not a language feature.

Which brings us to decorators (and I'm going to add context managers, as
its similar). Decorators (and even more so context managers) are used
commonly enough that we certainly have to introduce them in a intro class.
But there is a really big distinction between having some idea how to use
them, and knowing how they work / how to write them yourself.

So I introduce:

with open(filename) as the_file:
do_somethign_with(the_file)

early on, with only a hand-wavy explanation of what that with is all about.

And when i get to properties, I teach them:

@property
def a_method(self):


with also a hand-wavy explanation of what that @ symbol is.

and if := catches on, I expect it will be far more common that either of
those, especially in the simple script style codes that newbies are going
to be reading/writing at first.

But in the end, I, at least, am not trying to make the case that assignment
expressions are going to be particularly difficult to understand, but that
they are a significant complication, and any new feature like that makes
the language more complex. That's it.

I taught myself Python with version 1.5 in 1998 -- and have been teaching
it for I think almost ten years. In that time, the language has grown
substantially in complexity, and it does make it harder to teach.

We (UWPCE) recently revamped our 3-class program, and had a lot of debate
about how early to introduce things -- one of the instructors wanted to
leave off lambda and comprehensions 'till the second course in the
sequence, as part of functional programming. I I think he was right, if we
were teaching it in a more isolated environment, but both of those show up
in example code all over the place, so I've found it's necessary to
introduce a lot at a high level early:

decorators
comprehensions
lambda
context managers
*args and **kwargs
(off the top of my head)

So there is basically no way that we won't have to add assignment
expressions early on.

As someone else said: it's another straw on the haystack.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Matt Arcidy]

This cynical view on students is shocking!  Everyone on this list has
been a student or a learner for far longer than an educator, and the
perspective from students and learners are far more important than
educators to assess this angle regardless.  Can anyone adequately
explain why this specific modality of learning,  a student-in-a-seat
based educator, must outweigh all other modalities learners use to
increase knowledge and skill, from the perspectives of policy, tool
creation, and each of our time spent learning?

Shortest story:
Teach not to re-use names.

Short story:
1) What about the full mosaic of learning vs. this myopic view on
seat-based student-educator interaction?
2) What about smart, motivated, diligent and cautious students?
3) What weight should educator opinion be given with respect to
providing convenience to professional Python programmers?
4) Who is this Student Stupid von Densemeister anyways?
5) Are assignment expressions convenience and is any danger the pose
unmitagatble?
6) Consider adding an "Important Topics not Covered" or "Further
Reading" reading section to your class description
7) Creating examples showing this effect is easy, especially when not
actually re-using the name in the expression for explanatory purposes.
it's the same as creating examples showing how re-use works in
comprehensions.


Let's stop constructing these fake Students.  They only work as
appeals to the people we have come across whose lack of understanding
has made our life painful.  This construction is actively filtering
all the good students for the sake of influencing this decision, yet
again punishing or discounting the intelligent, quick, and diligent.

And what of this underlying premise that educator's should
_significantly_ influence language development?  Limiting Python's
tools to Student Straw-man's ability to learn is just dissonant, they
have nothing to do with each other, nor does this cause-effect
relationship actually exist.   Let's evaluate this reductionist
statement:
"I understand X, but this other person is not capable of understanding
X, therefore X should not exist"  Is has there ever been an X for
which this is true, let alone the backwardation necessary to fully
close the statement?

The actual argument is far less reductionist, yet even more ridiculous:
"I understand X,  this other person may take time to learn X, and may
use X wrong, therefore X should not exist"
"I understand assignment expressions, but this other class of person
may take time to learn assignment expressions, and may use assignment
expressions wrong, therefore assignment expressions should not be
accepted"

Rhetorically I disagree with how teaching is being presented, to the
point of near insult (for me lacking a better term).  You are saying
these statements about _my_ learning path, (though not personally of
course.)  Each of you occupied a role of student at some point, and
each of these statements are being made about your path as well.  Do
these ring true of your student experience?  What about your much
broader experience as a _learner_?  You think a tool shouldn't exist
because it took you time to learn it and you wrote some hard to debug
code, and possibly crashed production, got fired, lost your house and
your pet snake, and crashed the planet into the sun?

Now I yield, I will accept this position: all/some students cannot
learn this (or it's too complex to teach), but they must learn this
during some class to quickly become effective python developers.  How
much weight should this position have in this decision?  Let's appeal
to the learner in us.  How much of our learner's path, percentage of
total time learning all things python related, has been in a seat
listening to someone else, and that's the only place from which we
gained the knowledge to meet the educator's objective?  This time
spent in a class, how does that compare to hours in other learning
modalities?  Is this percentage not exactly the weight assigned to
that position?  Are people hired from pure class-room based experience
expected to require zero further learning?  Are people more valuable
based on classroom hours or work hours?

As for handling teaching the subject or not, this is easily remedied
with how I do it: "Important Topics not Covered", with resources.

Anyone here can rightfully claim educator status by having taught
another person something related to this language, which includes
at-work mentoring, informal discussions, posting/replying on SO,
blogging, etc.  Are they not being solicited to comment as well?  It's
possible to answer this question while vehemently disagreeing with the
PEP.  This focus on people who are being ostensibly paid to teach is
myopic.

Concretely, it's clear to me that parent-local effects can be
dangerously non-obvious when reading and mimicking code without
undertsanding.  But when?  And how to guard against?  How about this:
teach proper (i.e. not) re-using names.  The name will still be

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Michael Selik]

On Sun, Jul 1, 2018 at 5:28 PM Steven D'Aprano  wrote:

> On Sun, Jul 01, 2018 at 08:35:08AM -0700, Michael Selik wrote:
> > On Sun, Jul 1, 2018 at 12:39 AM Tim Peters  wrote:
> >
> > > So, ya, when someone claims [assignment expressions will] make Python
> > > significantly harder to teach, I'm skeptical of that claim.
> > >
> >
> > I don't believe anyone is making that claim. My worry is that assignment
> > expressions will add about 15 to 20 minutes to my class and a slight
> > discomfort.
>
> How do people who teach other languages deal with this?
>

Python may be in a unique situation in the history of programming. It
wouldn't surprise me if more people learned Python last year than any other
programming language.



> Assignment expressions are hardly a new-fangled innovation of Python's.
> They're used in Java, Javascript, Ruby, Julia, R, PHP and of course
> pretty much the entire C family (C, C++, C# at least). What do
> teachers of those languages do?
>

Assignment expressions are not the issue. The real question is: How do
open-source projects balance the addition of new features against the
growth of complexity? It's the same as that "Remember the Vasa" thread.


[...] R [has] *four* different ways of doing assignment.
>

I think that's a good explanation of why I teach Python and not R. The
first time someone asked me to teach a data science course, Python wasn't
the clear winner. In fact, R may have been more popular among
statisticians. I picked Python for the same reason it's more popular in the
industry -- it's the easiest* to use.

* Easiest that gets the job done well.


> As Mark and Chris said (quoting Mark below), this is just one straw in the
> > struggle against piling too many things on the haystack. Unlike some
> > changes to the language, this change of such general use that it won't be
> > an optional topic. Once widely used, it ain't optional.
>
> Without knowing the details of your course, and who they are aimed at,
> we cannot possibly judge this comment.


I disagree. I think the sentiment holds for a great variety of courses and
audiences.



> Decorators are widely used, but surely you don't teach them in a one day
> introductory class aimed at beginners?
>

Most of the time, no. Once, yes, because that's what the team needed. I was
pretty proud of myself for handling that one. Because I had to teach
decorators early, many other important topics were excluded.


Here is the syllabus for a ten week course:
> https://canvas.uw.edu/courses/1026775/pages/python-100-course-syllabus
>
> Note that decorators and even regular expressions don't get touched
> until week ten. If you can't fit assignment expressions in a ten week
> course, you're doing something wrong. If you can't fit them in a two
> hour beginners course, there is so much more that you aren't covering
> that nobody will notice the lack.
>

It's not about any one particular topic, but the trade-offs between topics.
A 10-week lecture course might be 30 hours of lecture, comparable to a
4-day "bootcamp" style course. I assure you that 4 days doesn't feel long
enough when those last few hours are winding down. There's always more to
say.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Sun, Jul 01, 2018 at 08:35:08AM -0700, Michael Selik wrote:
> On Sun, Jul 1, 2018 at 12:39 AM Tim Peters  wrote:
> 
> > So, ya, when someone claims [assignment expressions will] make Python
> > significantly harder to teach, I'm skeptical of that claim.
> >
> 
> I don't believe anyone is making that claim. My worry is that assignment
> expressions will add about 15 to 20 minutes to my class and a slight
> discomfort.

How do people who teach other languages deal with this?

Assignment expressions are hardly a new-fangled innovation of Python's. 
They're used in Java, Javascript, Ruby, Julia, R, PHP and of course 
pretty much the entire C family (C, C++, C# at least). What do 
teachers of those languages do?

R has a similar demographic of users (strong in the sciences, many 
beginners to programming, growing in popularity). Once R teachers have 
taught that you can assign values like this:

x = 1 + 2

does it take them 15-20 minutes to teach that you can do this as well?

   y = (x = 1 + 2) + 3

Admittedly R has the advantage that they don't have to teach a distinct 
assignment syntax and explain *why* it ought to be distinct. But 
countering that, they have *four* different ways of doing assignment.

x <- expression
expression -> x
x = expression
assign('x', expression)

(all of which can be used as expressions).


> As Mark and Chris said (quoting Mark below), this is just one straw in the
> struggle against piling too many things on the haystack. Unlike some
> changes to the language, this change of such general use that it won't be
> an optional topic. Once widely used, it ain't optional.

Without knowing the details of your course, and who they are aimed at, 
we cannot possibly judge this comment. Decorators are widely used, but 
surely you don't teach them in a one day introductory class aimed at 
beginners?

Here is the syllabus for a ten week course:

https://canvas.uw.edu/courses/1026775/pages/python-100-course-syllabus

Note that decorators and even regular expressions don't get touched 
until week ten. If you can't fit assignment expressions in a ten week 
course, you're doing something wrong. If you can't fit them in a two 
hour beginners course, there is so much more that you aren't covering 
that nobody will notice the lack.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Sun, Jul 1, 2018 at 8:35 AM, Michael Selik  wrote:

> As Mark and Chris said (quoting Mark below), this is just one straw in the
> struggle against piling too many things on the haystack. Unlike some
> changes to the language, this change of such general use that it won't be
> an optional topic. Once widely used, it ain't optional.
>

Exactly -- and I also emphasis that this would complicate the language in a
broad way -- a much bigger deal than adding a self contained new expression
or even the nonlocal keyword.

But to be clear about my take -- it will make the language that little bit
harder to teach, but it will also add a bit of complexity that will effect
us non-newbies as well.

Is it worth it? maybe. I know I like a better way to express the
loop-and-a-half concept in particular.

-CHB



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Michael Selik]

On Sun, Jul 1, 2018 at 12:39 AM Tim Peters  wrote:

> So, ya, when someone claims [assignment expressions will] make Python
> significantly harder to teach, I'm skeptical of that claim.
>

I don't believe anyone is making that claim. My worry is that assignment
expressions will add about 15 to 20 minutes to my class and a slight
discomfort.

As Mark and Chris said (quoting Mark below), this is just one straw in the
struggle against piling too many things on the haystack. Unlike some
changes to the language, this change of such general use that it won't be
an optional topic. Once widely used, it ain't optional.


On Sun, Jul 1, 2018 at 2:19 AM Mark Dickinson  wrote:

> There's a constant struggle to keep the Python portion of the course large
> enough to be coherent and useful, but small enough to allow time for the
> other topics.
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Mark Dickinson]

On Fri, Jun 22, 2018 at 7:28 PM, Chris Barker via Python-Dev <
python-dev@python.org> wrote:

>
> But once it becomes a more common idiom, students will see it in the wild
> pretty early in their path to learning python. So we'll need to start
> introducing it earlier than later.
>
> I think this reflects that the "smaller" a language is, the easier it is
> to learn.
>

For what it's worth, Chris's thoughts are close to my own here. I and
several of my colleagues teach week-long Python courses for Enthought. The
target audience is mostly scientists and data scientists (many of whom are
coming from MATLAB or R or IDL or Excel/VBA or some other development
environment, but some of whom are new to programming altogether), and our
curriculum is Python, NumPy, SciPy, Pandas, plus additional course-specific
bits and pieces (scikit-learn, NLTK, seaborn, statsmodels, GUI-building,
Cython, HPC, etc., etc.).

There's a constant struggle to keep the Python portion of the course large
enough to be coherent and useful, but small enough to allow time for the
other topics. To that end, we separate the Python piece of the course into
"core topics" that are essential for the later parts, and "advanced topics"
that can be covered if time allows, or if we get relevant questions. I
can't see a way that the assignment expression wouldn't have to be part of
the core topics. async stuff only appears in async code, and it's easy to
compartmentalize; in contrast, I'd expect that once the assignment
expression took hold we'd be seeing it in a lot of code, independent of the
domain.

And yes, I too see enough confusion with "is" vs == already, and don't
relish the prospect of teaching := in addition to those.

That's with my Python-teaching hat on. With my Python-developer hat on, my
thoughts are slightly different, but that's off-topic for this thread, and
I don't think I have anything to say that hasn't already been said many
times by others, so I'll keep quiet about that bit. :-)

-- 
Mark
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

I think I'll bow out of this now.  It's just too tedious.

Like here:

[Nick]
> I never said the motivation was to gain performance relative to the
> two-statement version - I said the motivation given in the PEP is to
> gain performance relative to the *repeated subexpression* version,
> *without* making the transition to the already supported two-statement
> version.

This is what you wrote:

>> "NAME := EXPR" exists on a different level of complexity, since it
>> adds name binding in arbitrary expressions for the sake of minor
>> performance improvement in code written by developers that are
>> exceptionally averse to the use of vertical screen real estate,

I'm not telepathic, so took it for it what said.  It didn't say a word
about "repeated expressions", nor a word about "two-statement versions"

It did say "minor" performance improvements, which sure suggested to me
that you had in mind the tiny changes in bytecode that can result from
squashing a current two-statement spelling into a one-statement form with
an embedded assignment.  That fit both "minor performance improvement" and
"vertical screen space", which were the only the two clues I had to go on.

In the example you eventually gave in a later message, the performance
difference was more on the order of a factor 2 (regexp searches can be
expensive indeed), which matches nobody's idea of "minor".  So the use of
"minor" wholly ruled out in my mind that you had _anything_ akin to your
eventual example in mind.

For the rest, sure, words could be added to the PEP without end.   At least
some of the points I covered were telegraphically mentioned in my Appendix,
although I'll grant that nobody else is telepathic either ;-)

For the question:


> > The remaining point of contention is only the "Inevitable cost of

> > change" one: given the level of disruption this will cause in the way

> > that Python gets taught to new users, is it giving a commensurate

> > pay-off in increased semantic expressiveness?

> >

> > My answer to that question remains "No", while your answer is either

> > "Yes" or "I don't see why that should matter" (I'm genuinely unsure

> > which).

I don't know, and I'm not qualified to guess - I don't teach Python to new
users for a living.  Decades ago I tutored "advanced" engineering
undergrads in a variety of science-y subjects, and was routinely surprised
by what broke their brains.

I have noted that assignment expressions have been part of a great many
languages for decades (this isn't cutting edge tech), questions about them
are conspicuous by absence on StackOverflow (everyone else seems to teach
them effectively), and skimming various online teaching materials for other
languages convinced me that none of those professional educators thought it
needed more than a page to teach (and to teach them with relatively few
words).  There's really not much to them:  people have to learn what
binding means in Python regardless, pretty much starting in the first hour,
yes?  "Binding" on its own is the hard part.

If they don't drop out and stick around for the 10-minute lesson on
assignment expressions 3 weeks later, will _that_ finally break their
brains?  "Wow - it not only binds the name, but ALSO returns the object
that was bound?!  I just can't take it - please, let's go back to the
lesson on metaclasses" just doesn't seem likely to me ;-)

At heart, ":=" could make a good case for being the simplest of all
Python's operators.  It does no computation at all, and you don't even have
to worry about a dunder method changing its meaning depending on context.

So, ya, when someone claims they'll make Python significantly harder to
teach, I'm skeptical of that claim.  Which does not mean I believe it's
wrong - it means I'm skeptical.  I would also be skeptical of a claim that
teaching them would be no trouble at all, except nobody has made such a
claim ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Sat, Jun 30, 2018 at 11:32:03PM -0500, Tim Peters wrote:

[...]
> So, no, gaining performance is _not_ the motivation here.  You already had
> a way to make it "run fast'.  The motivation is the _brevity_ assignment
> expressions allow while _retaining_ all of the two-statement form's
> advantages in easier readability, easier reasoning, reduced redundancy, and
> performance.

I think the two of you (Tim and Nick) are in violent agreement :-)

In fairness to Nick performance is _a_ motivation here, in the sense of 
"how can we avoid making redundant multiple calls to a function in an 
expression without splitting it into multiple statements?".

If performance were the *only* concern, then I agree with your point 
that we already have a solution. Simply refactor this:

process(expensive_call(arg), expensive_call(arg) + 1)

to this:

useful_value = expensive_call(arg)
process(useful_value, useful_value + 1)

But since performance is not the *only* concern, all the other factors 
you refer to (saving vertical space, DRY, side-effects, etc) count too.

Assignment expressions aren't valuable because they give us One Big Win. 
They're valuable because they give us Many Little Wins, which we 
(proponents of PEP 572) believe will outweigh the (minor) additional 
costs in complexity and opportunities for abuse.

The closest (in my opinion) assignment expressions comes to a One Big 
Win is to reduce the need for cascades of if... statements:

m = re.match(pattern, text)
if m:
...
else:
m = re.match(other_pattern, text)
if m:
...
else:
m = re.match(third_pattern, text)
if m: ...


which wastes both vertical and horizontal space.

If that were the *only* win, I'd consider dedicated syntax for if/elif 
statements alone, but it isn't. Little wins include:

- use in while statement headers, "while x:= expr"

- DRY inside comprehensions, as an alternative to the neat but somewhat
  obscure idiom:

[(x, x+1, x*2) for a in it for x in (func(a),) if x]

- running totals and similar inside comprehensions, where
  we need a way to initialise the total on the first iteration

total = 0
[total := total + a for a in it]

- and examples such as Tim's use of Heron's Formula:

area = sqrt((s := (a+b+c)/2)*(s-a)*(s-b)*(s-c))

This last one is, I think, the only one where the postfix form 
reads better, but maybe I only say that because I'm more 
familiar with it:

area = sqrt(s*(s-a)*(s-b)*(s-c)) where s = (a+b+c)/2



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 1 July 2018 at 14:32, Tim Peters  wrote:
> [Nick]
>
>> The PEP specifically cites this example as motivation:
>
> The PEP gives many examples.  Your original was a strawman
> mischaracterization of the PEP's _motivations_ (note the plural:  you only
> mentioned "minor performance improvement", and snipped my listing of the
> major motivations).

I listed two motivations, not one:

1. Minor performance improvements (the "avoiding repeated
subexpressions without use multiple statements" rational)
2. Making certain coding patterns easier to spot (the loop-and-a-half
and if-elif chaining cases)

Technically, avoid repeated subexpressions without requiring a
separate line also falls into the second category.

The subsequent interaction with comprehensions and generator
expressions is an interesting side effect of extending the basic idea
to a fully coherent and self-consistent proposal, not one of the
original motivations for it.

>>   group = re.match(data).group(1) if re.match(data) else None
>
>> That code's already perfectly straightforward to read and write as a
>
>> single line,
>
> I disagree.  In any case of textual repetition, it's a visual
> pattern-matching puzzle to identify the common substrings (I have to
> visually scan that line about 3 times to be sure), and then a potentially
> difficult conceptual puzzle to figure out whether side effects may result in
> textually identical substrings evaluating to different objects.  That's why
> "refererential transparency" is so highly valued in functional languages
> ("if subexpressions are spelled the same, they evaluate to the same result,
> period" - which isn't generally true in Python - to get that enormously
> helpful (to reasoning) guarantee in Python you have to ensure the
> subexpression is evaluated exactly once).
>
> And as you of all people should be complaining about, textual repetition is
> also prone to "oops - forgot one!" and "oops! made a typo when changing the
> second one!" when code is later modified.

That's a reasonable readability based argument, but it's not what the
PEP currently gives as a motivation for this aspect of the proposal.

>> so the only reason to quibble about it
>
> I gave you three better reasons to quibble about it just above ;-)

Then add them to the PEP, as what's currently there really isn't
offering a compelling motivation for this aspect of the proposal :)

>> is because it's slower than the arguably less clear two-line alternative:
>
>>  _m = re.match(data)
>
>>   group = _m.group(1) if _m else None
>
>
> I find that much clearer than the one-liner above:  the visual pattern
> matching is easier because the repeated substring is shorter and of much
> simpler syntactic structure; it guarantees _by construction_ that the two
> instances of `_m` evaluate to the same object, so there's no possible
> concern about that (it doesn't even matter if you bound `re` to some
> "non-standard" object that has nothing to do with Python's `re` module); and
> any later changes to the single instance of `re.match(data)` don't have to
> be repeated verbatim elsewhere.  It's possible that it runs twice as fast
> too, but that's the least of my concerns.

I agree with this, but also think the two-line form is a perfectly
acceptable way of spelling it, and a perfectly acceptable refactoring
of the one-line form with duplicated subexpressions to improve
maintainability.

> All of those advantages are retained in the one-liner too if an assignment
> expression can be used in it.

Sure, but the open design question is whether folks that would have
written the one-liner with repeated subexpressions are going to be any
more likely to use an assignment expression to avoid the repetition
without prompting by a more experienced developer than they are to use
a separate preceding assignment statement.

That assessment of "What is the increased chance that the repeated
subexpression will be avoided when the code is first written?" then
gets traded off against the overall increase in language complexity
arising from allowing name bindings in arbitrary subexpressions.

Don't get me wrong, I now agree that the proposal in PEP 572 is the
most coherent and self-consistent approach to assignment expressions
that we could pursue given the existing scoping semantics of
comprehensions and generator expressions.

The remaining point of contention is only the "Inevitable cost of
change" one: given the level of disruption this will cause in the way
that Python gets taught to new users, is it giving a commensurate
pay-off in increased semantic expressiveness?

My answer to that question remains "No", while your answer is either
"Yes" or "I don't see why that should matter" (I'm genuinely unsure
which).

>>> sometimes it allows a more compact way of reusing an expensive
>>> subexpression by giving it a name.   Which they already do by giving
>>> it a name in a separate statement, so the possible improvement would
>>> be in brevity rather than 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Nick Coghlan wrote:

That's a performance argument, not a readability one (as if you don't
care about performance, you can just repeat the subexpression).


Repeated subexpressions can be a readability issue too, since you
have to examine them to notice they are actually the same. They
also provide an opportunity to make the error of not making them
the same when they should be, and add the maintenance burden
of ensuring they stay the same when changes are made.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Nick Coghlan]

>>> "NAME := EXPR" exists on a different level of complexity, since it
>>> adds name binding in arbitrary expressions for the sake of minor
>>> performance improvement in code written by developers that are
>>> exceptionally averse to the use of vertical screen real estate,

> >>> ...

[Tim]
>> Note that PEP 572 doesn't contain a single word about "performance"
(neither

> >> that specific word nor any synonym), and I gave only one thought to it
when

> >> writing Appendix A:  "is this going to slow anything down
significantly?".

> >> The answer was never "yes", which I thought was self-evident, so I never

> >> mentioned it.  Neither did Chris or Guido.

> >>

> >> Best I can recall, nobody has argued for it on the grounds of
"performance".

> >> except in the indirect sense that sometimes it allows a more compact
way of

> >> reusing an expensive subexpression by giving it a name.   Which they
already

> >> do by giving it a name in a separate statement, so the possible
improvement

> >> would be in brevity rather than performance.

[Nick]

> > The PEP specifically cites this example as motivation:

The PEP gives many examples.  Your original was a strawman
mischaracterization of the PEP's _motivations_ (note the plural:  you only
mentioned "minor performance improvement", and snipped my listing of the
major motivations).

>
>   group = re.match(data).group(1) if re.match(data) else None

> >

> > That code's already perfectly straightforward to read and write as a

> > single line,

I disagree.  In any case of textual repetition, it's a visual
pattern-matching puzzle to identify the common substrings (I have to
visually scan that line about 3 times to be sure), and then a potentially
difficult conceptual puzzle to figure out whether side effects may result
in textually identical substrings evaluating to different objects.  That's
why "refererential transparency" is so highly valued in functional
languages ("if subexpressions are spelled the same, they evaluate to the
same result, period" - which isn't generally true in Python - to get that
enormously helpful (to reasoning) guarantee in Python you have to ensure
the subexpression is evaluated exactly once).

And as you of all people should be complaining about, textual repetition is
also prone to "oops - forgot one!" and "oops! made a typo when changing the
second one!" when code is later modified.

> so the only reason to quibble about it

I gave you three better reasons to quibble about it just above ;-)


> is because it's slower than the arguably less clear two-line alternative:

> >

> >  _m = re.match(data)

> >   group = _m.group(1) if _m else None

>
I find that much clearer than the one-liner above:  the visual pattern
matching is easier because the repeated substring is shorter and of much
simpler syntactic structure; it guarantees _by construction_ that the two
instances of `_m` evaluate to the same object, so there's no possible
concern about that (it doesn't even matter if you bound `re` to some
"non-standard" object that has nothing to do with Python's `re` module);
and any later changes to the single instance of `re.match(data)` don't have
to be repeated verbatim elsewhere.  It's possible that it runs twice as
fast too, but that's the least of my concerns.

All of those advantages are retained in the one-liner too if an assignment
expression can be used in it.

> Thus the PEP's argument is that it wants to allow the faster version

> > to remain a one-liner that preserves the overall structure of the

> > version that repeats the subexpression:

> >

> > group = _m.group(1) if _m := re.match(data) else None

> >

> > That's a performance argument, not a readability one (as if you don't

> > care about performance, you can just repeat the subexpression).

>
How does that differ from the part of what I said that you did retain above?

>> sometimes it allows a more compact way of reusing an expensive
>> subexpression by giving it a name.   Which they already do by giving
>> it a name in a separate statement, so the possible improvement would
>> be in brevity rather than performance.

You already realized the performance gain could be achieved by using two
statements.  The _additional_ performance gain by using assignment
expressions is at best trivial (it may save a LOAD_FAST opcode to fetch the
object bound to `_m` for the `if` test).

So, no, gaining performance is _not_ the motivation here.  You already had
a way to make it "run fast'.  The motivation is the _brevity_ assignment
expressions allow while _retaining_ all of the two-statement form's
advantages in easier readability, easier reasoning, reduced redundancy, and
performance.

As Guido said, in the PEP, of the example you gave here:

Guido found several examples where a programmer repeated
a subexpression, slowing down the program, in order to save
one line of code


It couldn't possibly be clearer that Guido thought the programmer's
motivation was brevity ("in 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 1 July 2018 at 02:37, Tim Peters  wrote:
> [Nick Coghlan]
>
>> ...
>
>> "NAME := EXPR" exists on a different level of complexity, since it
>
>> adds name binding in arbitrary expressions for the sake of minor
>
>> performance improvement in code written by developers that are
>
>> exceptionally averse to the use of vertical screen real estate,
>> ...
>
> Note that PEP 572 doesn't contain a single word about "performance" (neither
> that specific word nor any synonym), and I gave only one thought to it when
> writing Appendix A:  "is this going to slow anything down significantly?".
> The answer was never "yes", which I thought was self-evident, so I never
> mentioned it.  Neither did Chris or Guido.
>
> Best I can recall, nobody has argued for it on the grounds of "performance".
> except in the indirect sense that sometimes it allows a more compact way of
> reusing an expensive subexpression by giving it a name.   Which they already
> do by giving it a name in a separate statement, so the possible improvement
> would be in brevity rather than performance.

The PEP specifically cites this example as motivation:

   group = re.match(data).group(1) if re.match(data) else None

That code's already perfectly straightforward to read and write as a
single line, so the only reason to quibble about it is because it's
slower than the arguably less clear two-line alternative:

   _m = re.match(data)
   group = _m.group(1) if _m else None

Thus the PEP's argument is that it wants to allow the faster version
to remain a one-liner that preserves the overall structure of the
version that repeats the subexpression:

   group = _m.group(1) if _m := re.match(data) else None

That's a performance argument, not a readability one (as if you don't
care about performance, you can just repeat the subexpression).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Terry Reedy]

On 6/30/2018 5:35 AM, Steven D'Aprano wrote:


I've given reasons why I believe that people will expect assignments in
comprehensions to occur in the local scope. Aside from the special case
of loop variables, people don't think of comprehensions as a separate
scope.


I think this is because comprehensions other than generator expressions 
were originally *defined* in terms of equivalent code in the *same* 
local scope, are still easily thought of in those terms, and, as I 
explained in my response to Guido, could, at least in simple cases, 
still be implemented in the local scope, so that assignment expressions 
would be executed and affect the expected local scope without 'nonlocal'.


Generator expressions, on the other hand, have always been defined in 
terms of equivalent code in a *nested* scope, and must be implemented 
that way, so some re-definition and re-implementation is needed for 
assignment expressions to affect the local scope in which the g.e is
defined and for that effect to be comprehensible.  It might be enough to 
add something like "any names that are targets of assignment expressions 
are added to global or nonlocal declarations within the implementation 
generator function."


If the equality [generator-expression] == list(generator-expression] is 
preserved, then it could serve as the definition of the list 
comprehension.  The same could be true for set and dict comprehensions, 
with the understanding that the equality is modified to {a:b for ...} == 
dict((a,b) for ...).  It should also be mentioned that the defining 
equality is not necessarily the implementation.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Michael Selik]

On Sat, Jun 30, 2018 at 9:43 AM Tim Peters  wrote:

> The attractions are instead in the areas of reducing redundancy, improving
> clarity, allowing to remove semantically pointless indentation levels in
> some cases, indeed trading away some horizontal whitespace in otherwise
> nearly empty lines for freeing up a bit of vertical screen space, and in
> the case of comprehensions/genexps adding straightforward ways to
> accomplish some conceptually trivial things that at best require trickery
> now (like emulating a cell object by hand).
>

The examples you provided (some were new in this thread, I think) are
compelling. While my initial reaction to the proposal was mild horror, I'm
not troubled by the scoping questions.

Issues still bothering me:
1. Initial reactions from students was confusion over := vs =
2. This seems inconsistent with the push for type hints

To be fair, I felt a similar gut reaction to f-strings, and now I can't
live without them. Have I become a cranky old man, resistant to change?
Your examples have put me into the "on the fence, slightly worried"
category instead of "clearly a bad idea".

On scoping, beginners seem more confused by UnboundLocalError than by
variables bleeding between what they perceive as separate scopes. The
concept of a scope can be tricky to communicate. Heck, I still make the
mistake of looking up class attributes in instance methods as if they were
globals. Same-scope is natural. Natural language is happy with ambiguity.
Separate-scope is something programmers dreamed up. Only experienced C,
Java, etc. programmers get surprised when they make assumptions about what
syntax in Python creates separate scopes, and I'm not so worried about
those folks. I remind them that the oldest versions of C didn't have block
scopes (1975?) and they quiet down.

The PEP lists many exclusions of where the new := operator is invalid [0].
I unfortunately didn't have a chance to read the initial discussion over
the operator. I'm sure it was thorough :-). What I can observe is that each
syntactical exclusion was caused by a different confusion, probably teased
out by that discussion. Many exclusions means many confusions.

My intuition is that the awkwardness stems from avoiding the replacement of
= with :=. Languages that use := seem to avoid the Yoda-style comparison
recommendation that is common to languages that use = for assignment
expressions. I understand the reluctance for such a major change to the
appearance of Python code, but it would avoid the laundry list of
exclusions. There's some value in parsimony.

Anyway, we've got some time for testing the idea on live subjects.

Have a good weekend, everyone.
-- Michael

PS. Pepe just tied it up for Portugal vs Uruguay. Woo! ... and now Cavani
scored again :-(

[0] https://www.python.org/dev/peps/pep-0572/#exceptional-cases
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Nick Coghlan]

> > ...

> > "NAME := EXPR" exists on a different level of complexity, since it

> > adds name binding in arbitrary expressions for the sake of minor

> > performance improvement in code written by developers that are

> > exceptionally averse to the use of vertical screen real estate,
> ...

Note that PEP 572 doesn't contain a single word about "performance"
(neither that specific word nor any synonym), and I gave only one thought
to it when writing Appendix A:  "is this going to slow anything down
significantly?".  The answer was never "yes", which I thought was
self-evident, so I never mentioned it.  Neither did Chris or Guido.

Best I can recall, nobody has argued for it on the grounds of
"performance". except in the indirect sense that sometimes it allows a more
compact way of reusing an expensive subexpression by giving it a name.
 Which they already do by giving it a name in a separate statement, so the
possible improvement would be in brevity rather than performance.

The attractions are instead in the areas of reducing redundancy, improving
clarity, allowing to remove semantically pointless indentation levels in
some cases, indeed trading away some horizontal whitespace in otherwise
nearly empty lines for freeing up a bit of vertical screen space, and in
the case of comprehensions/genexps adding straightforward ways to
accomplish some conceptually trivial things that at best require trickery
now (like emulating a cell object by hand).

Calling all that "for the sake of minor performance improvements" - which
isn't even in the list -  is so far off base it should have left me
speechless - but it didn't ;-)

But now that you mention it, ya, there will be a trivial performance
improvement in some cases.  I couldn't care less about that, and can
confidently channel that Guido doesn't either.  It would remain fine by me
if assignment expressions ran trivially slower.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 30 June 2018 at 19:35, Steven D'Aprano  wrote:
> So I think Q1 is the critical one. And I think the answer is, no,
> they're conceptually bloody simple. They evaluate the expression on the
> right, assign it to the name on the left, and return that value.

And the proposed parent local scoping in PEP 572 has the virtue of
making all of the following do *exactly the same thing*, just as they
would in the version without the assignment expression:

ref = object()
container = [item := ref]
container = [x for x in [item := ref]]
container = [x for x in [item := ref] for i in range(1)]
container = list(x for x in [item := ref])
container = list(x for x in [item := ref] for i in range(1))

# All variants pass this assertion, keeping the implicit sublocal
scope almost entirely hidden
assert container == [ref] and item is ref and item is container[0]

My own objections were primarily based on the code-generation-centric
concept of wanting Python's statement level scoping semantics to
continue to be a superset of its expression level semantics, and
Guido's offer to define "__parentlocal" in the PEP as a conventional
shorthand for describing comprehension style assignment expression
scoping when writing out their statement level equivalents as
pseudo-code addresses that. (To put it another way: it turned out it
wasn't really the semantics themselves that bothered me, since they
have a lot of very attractive properties as shown above, it was the
lack of a clear way of referring to those semantics other than "the
way assignment expressions behave in implicit comprehension and
generator expression scopes").

Cheers,
Nick.



-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Thu, Jun 28, 2018 at 03:42:49PM -0700, Chris Barker via Python-Dev wrote:

> If we think hardly anyone is ever going to do that -- then I guess it
> doesn't matter how it's handled.

That's how you get a language with surprising special cases, gotchas and 
landmines in its behaviour. (Cough PHP cough.)

It is one thing when gotchas occur because nobody thought of them, or 
because there is nothing you can do about them. But I do not think it is 
a good idea to *intentionally* leave gotchas lying around because "oh, I 
didn't think anyone would ever do that...".

*wink*


[...]
> but here the keyword "nonlocal" is used -- you are clearly declaring that
> you are messing with a nonlocal name here -- that is a lot more obvious
> than simply using a :=

But from the point of view of somebody reading the code, there is no 
need for a nonlocal declaration, since the assignment is just a local 
assignment.

Forget the loop variable -- it is a special case where "practicality 
beats purity" and it makes sense to have it run in a sublocal scope. 
Everything else is just a local, regardless of whether it is in a 
comprehension or not.


> And "nonlocal" is not used that often, and when it is it's for careful
> closure trickery -- I'm guessing := will be far more common. And, of
> course, when a newbie encounters it, they can google it and see what it
> means -- far different that seeing a := in a comprehension and
> understanding (by osmosis??) that it might make changes in the local scope.

I've given reasons why I believe that people will expect assignments in 
comprehensions to occur in the local scope. Aside from the special case 
of loop variables, people don't think of comprehensions as a separate 
scope.

There's no Comprehension Sublocal-Local-Enclosing Local-Global-Builtin 
scoping rule. (Do you want there to be?) Even *class scope* comes as an 
unfamiliar surprise to people.

I do not believe that people will "intuitively" expect assignments in a 
comprehension to disappear when the comprehension finishes -- I expect 
that most of the time they won't even think about it, but when they do, 
they'll expect it to hang around like *every other use* of assignment 
expressions.


> And  I don't think you can even do that with generator expressions now --
> as they can only contain expressions.

It makes me cry to think of the hours I spent -- and the brownie points 
I lost with my wife -- showing how you can already simulate this with 
locals() or globals(). Did nobody read it? :-(

https://mail.python.org/pipermail/python-dev/2018-June/154114.html

Yes, you can do this *right now*. We just don't because playing around 
with locals() is a dodgy thing to do.


> Maybe it's only comprehensions, and maybe it'll be rare to have a confusing
> version of those, so it'll be no big deal, but this thread started talking
> about educators' take on this -- and as an educator, I think this really
> does complicate the language.

See my recent post here:

https://mail.python.org/pipermail/python-dev/2018-June/154184.html

I strongly believe that the "comprehensions are local, like everything 
else" scenario is simpler and less surprising and easier to explain than 
hiding assignments inside a sublocal comprehension scope that hardly 
anyone even knows exists.

Especially if we end up doing it inconsistently and let variables 
sometimes leak.


> Python got much of it's "fame" by being "executable pseudo code" -- its
> been moving farther and farther away from those roots. That's generally a
> good thing, as we've gain expressiveness in exchangel, but we shouldn't
> pretend it isn't happening, or that this proposal doesn't contribute to
> that trend.

I think there are two distinct forms of "complication" here.

1. Are assignment expressions in isolation complicated?

2. Given assignment expressions, can people write obfuscated,
   complex code?


Of course the answer to Q2 is yes, the opportunity will be there. 
Despite my general cynicism about my fellow programmers, I actually do 
believe that the Python community does a brilliant job of self-policing 
to prevent the worst excesses of obfuscatory one-liners. I don't think 
that will change.

So I think Q1 is the critical one. And I think the answer is, no, 
they're conceptually bloody simple. They evaluate the expression on the 
right, assign it to the name on the left, and return that value.

Here is a question and answer:

Question: after ``result = (x := 2) + 3``, what is the value of x?

Answer: 2.

Question: what if we put the assignment inside a function call? 
``f((x:=2), x+3)``

Answer: still 2.

Question: how about inside a list display? ``[1, x:=2, 3]``

Answer: still 2.

Question: what about a dict display? ``{key: x:=2}`` A tuple? A set?

Answer: still 2 to all of them.

Question: how about a list comprehension?

Answer: ah, now, that's complicated, it depends on which bit of the 
comprehension you put it in, 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Sat, Jun 30, 2018 at 06:30:56PM +1000, Nick Coghlan wrote:

> The significant semantic differences between "{x : 1}" and "{x := 1}"
> are also rather surprising :)

*Significant* and obvious differences are good. It's the subtle 
differences that you don't notice immediately that really hurt:

{x+1} versus {x-1}
x > y versus x < y
x/y versus x//y

alist = [a, b]
alist = (a, b)

Sometimes small differences in punctuation or spelling make a big 
difference to semantics.


Punctuation Saves Lives!

"Let's eat, grandma!"

"Let's eat grandma!"

Unless you propose to ban all operators and insist on a minimum string 
distance between all identifiers:

https://docs.python.org/3/library/os.html#os.spawnl

picking out little differences in functionality caused by little 
differences in code is a game we could play all day.

At least we won't have the "=" versus "==" bug magnet from C, or the 
"==" versus "===" confusion from Javascript. Compared to that, the 
in-your-face obvious consequences of {x: 1} versus {x := 1} are pretty 
harmless.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 30 June 2018 at 09:49, Chris Barker - NOAA Federal via Python-Dev
 wrote:
>> On Jun 28, 2018, at 8:21 PM, Tim Peters  wrote:
>
> Seems it’s all been said, and Tim’s latest response made an excellent
> case for consistency.
>
> But:
>
>> Regardless of how assignment expressions work in listcomps and genexps, this 
>> example (which uses neither) _will_ rebind the containing block's `x`:
>>
>> [x := 1]
>
> This reinforces my point that it’s not just about comprehensions, but
> rather that the local namespace can be altered anywhere an expression
> is used  — which is everywhere.
>
> That trivial example is unsurprising, but as soon as your line of code
> gets a bit longer, it could be far more hidden.

The significant semantic differences between "{x : 1}" and "{x := 1}"
are also rather surprising :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 29 June 2018 at 08:42, Chris Barker via Python-Dev
 wrote:
> On Thu, Jun 28, 2018 at 9:28 AM, Tim Peters  wrote:
>> Did adding ternary `if` (truepart if expression else falsepart) complicate
>> the language significantly?
>
>
> I don't think so -- no. For two reasons:
>
> 1) the final chosen form is kind of verbose, but therefor more like
> "executable pseudo code" :-) As apposed to the C version, for instance.
>
> 2) it added one new construct, that if, when someone sees it for the first
> (or twenty fifth) time and doesn't understand it, they can look it up, and
> find out. and it only effects that line of code.
>
> So adding ANYTHING does complicate the language, by simply making it a bit
> larger, but some things are far more complicating than others.

It's worth noting that without the bug prone "C and A or B" construct
(which gives the wrong result when "not A" is True), we'd likely never
have gotten "A if C else B" (which gives the right result regardless
of the truth value of A). In the case of PEP 308, the new construction
roughly matched the existing idiom in expressive power, it just
handled it correctly by being able to exactly match the developer's
intent.

"NAME := EXPR" exists on a different level of complexity, since it
adds name binding in arbitrary expressions for the sake of minor
performance improvement in code written by developers that are
exceptionally averse to the use of vertical screen real estate, and
making a couple of moderately common coding patterns (loop-and-a-half,
if-elif-chains with target binding) more regular, and hence easier to
spot.

I think the current incarnation of PEP 572 does an excellent job of
making the case that says "If we add assignment expressions, we should
add them this particular way" - there are a lot of syntactic and
semantic complexities to navigate, and it manages to make its way
through them and still come out the other side with a coherent and
self-consistent proposal that copes with some thoroughly quirky
existing scoping behaviour.

That only leaves the question of "Does the gain in expressive power
match the increase in the cognitive burden imposed on newcomers to the
language?", and my personal answer to that is still "No, I don't think
it does". It isn't my opinion on that that matters, though: I think
that's now going to be a conversation between Guido and folks that are
actively teaching Python to new developers, and are willing to get
their students involved in some experiments.

Cheers,
Nick.

P.S. It does make me wonder if it would be possible to interest the
folks behind https://quorumlanguage.com/evidence.html in designing and
conducting fully controlled experiments comparing the
comprehensibility of pre-PEP-572 code with post-PEP-572 code *before*
the syntax gets added to the language :)

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 09:52:43PM -0700, Chris Barker wrote:

> It seems everyone agrees that scoping rules should be the same for
> generator expressions and comprehensions,

Yes. I dislike saying "comprehensions and generator expressions" over 
and over again, so I just say "comprehensions".


Principle One:

- we consider generator expressions to be a lazy comprehension;
- or perhaps comprehensions are eager generator expressions;
- either way, they behave the same in regard to scoping rules.


Principle Two:

- the scope of the loop variable stays hidden inside the 
  sub-local ("comprehension" or "implicit hidden function")
  scope;
- i.e. it does not "leak", even if you want it to.


Principle Three:

- glossing over the builtin name look-up, calling list(genexpr)
  will remain equivalent to using a list comprehension;

- similarly for set and dict comprehensions.


Principle Four:

- comprehensions (and genexprs) already behave "funny" inside
  class scope; any proposal to fix class scope is beyond the,
  er, scope of this PEP and can wait for another day.


So far, there should be (I hope!) no disagreement with those first four 
principles. With those four principles in place, teaching and using 
comprehensions (genexprs) in the absense of assignment expressions does 
not need to change one iota.

Normal cases stay normal; weird cases mucking about with locals() inside 
the comprehension are already weird and won't change.


> So what about:
> 
> l = [x:=i for i in range(3)]
> 
> vs
> 
> g = (x:=i for i in range(3))
> 
> Is there any way to keep these consistent if the "x" is in the regular
> local scope?

Yes. That is what closures already do.

We already have such nonlocal effects in Python 3. Move the loop inside 
an inner (nested) function, and then either call it immediately to 
simulate the effect of a list comprehension, or delay calling it to 
behave more like a generator expression.

Of course the *runtime* effects depend on whether or not the generator 
expression is actually evaluated. But that's no mystery, and is 
precisely analogous to this case:

def demo():
x = 1
def inner():
nonlocal x
x = 99
inner()  # call the inner function
print(x)


This prints 99. But if you comment out the call to the inner function, 
it prints 1. I trust that doesn't come as a surprise.

Nor should this come as a surprise:

def demo():
x = 1
# assuming assignment scope is local rather than sublocal
g = (x:= i for i in (99,))
L = list(g)
print(x)

The value of x printed will depend on whether or not you comment out 
the call to list(g).


> Note that this thread is titled "Informal educator feedback on PEP 572".
> 
> As an educator -- this is looking harder an harder to explain to newbies...
> 
> Though easier if any assignments made in a "comprehension" don't "leak out".

Let me introduce two more principles.


Principle Five:

- all expressions are executed in the local scope.


Principle Six:

- the scope of an assignment expression variable inside a
  comprehension (genexpr) should not depend on where inside
  the comprehension it sits.


Five is, I think, so intuitive that we forget about it in the same way 
that we forget about the air we breathe. It would be surprising, even 
shocking, if two expressions in the same context were executed in 
different scopes:

result = [x + 1, x - 2]

If the first x were local and the second was global, that would be 
disturbing. The same rule ought to apply if we include assignment 
expressions:

result = [(x := expr) + 1, x := x - 2]

It would be disturbing if the first assignment (x := expr) executed in 
the local scope, and the second (x := x - 2) failed with NameError 
because it was executed in the global scope.

Or worse, *didn't* fail with NameError, but instead returned something 
totally unexpected.

Now bring in a comprehension:

result = [(x := expr) + 1] + [x := x - 2 for a in (None,)]

Do you still want the x inside the comprehension to be a different x to 
the one outside the comprehension? How are you going to explain that 
UnboundLocalError to your students?

That's not actually a rhetorical question. I recognise that while 
Principle Five seems self-evidently desirable to me, you might consider 
it less important than the idea that "assignments inside comprehensions 
shouldn't leak".

I believe that these two expressions should give the same results even 
to the side-effects:

[(x := expr) + 1, x := x - 2]

[(x := expr) + 1] + [x := x - 2 for a in (None,)]

I think that is the simplest and most intuitive behaviour, the one 
which will be the least surprising, cause the fewest unexpected 
NameErrors, and be the simplest to explain.

If you still prefer the "assignments shouldn't leak" idea, consider 
this: under the current implementation of comprehensions as an implicit 
hidden function, the scope of a variable depends on *where* it is, 
violating Principle Six.

(That was the 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 28 June 2018 at 08:31, Guido van Rossum  wrote:
> So IIUC you are okay with the behavior described by the PEP but you want an
> explicit language feature to specify it?
>
> I don't particularly like adding a `parentlocal` statement to the language,
> because I don't think it'll be generally useful. (We don't have `goto` in
> the language even though it could be used in the formal specification of
> `if`, for example. :-)
>
> But as a descriptive mechanism to make the PEP's spec clearer I'm fine with
> it. Let's call it `__parentlocal` for now. It would work a bit like
> `nonlocal` but also different, since in the normal case (when there's no
> matching `nonlocal` in the parent scope) it would make the target a local in
> that scope rather than trying to look for a definition of the target name in
> surrounding (non-class, non-global) scopes. Also if there's a matching
> `global` in the parent scope, `__parentlocal` itself changes its meaning to
> `global`. If you want to push a target through several level of target
> scopes you can do that by having a `__parentlocal` in each scope that it
> should push through (this is needed for nested comprehensions, see below).
>
> Given that definition of `__parentlocal`, in first approximation the scoping
> rule proposed by PEP 572 would then be: In comprehensions (which in my use
> in the PEP 572 discussion includes generator expressions) the targets of
> inline assignments are automatically endowed with a `__parentlocal`
> declaration, except inside the "outermost iterable" (since that already runs
> in the parent scope).
>
> There would have to be additional words when comprehensions themselves are
> nested (e.g. `[[a for a in range(i)] for i in range(10)]`) since the PEP's
> intention is that inline assignments anywhere there end up targeting the
> scope containing the outermost comprehension. But this can all be expressed
> by adding `__parentlocal` for various variables in various places (including
> in the "outermost iterable" of inner comprehensions).
>
> I'd also like to keep the rule prohibiting use of the same name as a
> comprehension loop control variable and as an inline assignment target; this
> rule would also prohibit shenanigans with nested comprehensions (for any set
> of nested comprehensions, any name that's a loop control variable in any of
> them cannot be an inline assignment target in any of them). This would also
> apply to the "outermost iterable".
>
> Does this help at all, or did I miss something?

Yep, it does, and I don't think you missed anything.

Using "__parentlocal" to indicate "parent local scoping semantics
apply here" still gives the concept a name and descriptive shorthand
for use in pseudo-code expansions of assignment expressions in
comprehensions, without needing to give it an actually usable
statement level syntax, similar to the way we use "_expr_result" and
"_outermost_iter" to indicate references that in reality are entries
in an interpreter's stack or register set, or else a pseudo-variable
that doesn't have a normal attribute identifier.

And if anyone does want to make the case for the syntax being
generally available, they don't need to specify how it should work -
they just need to provide evidence of cases where it would clarify
code unrelated to the PEP 572 use case.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Tim]
>> Regardless of how assignment expressions work in listcomps and genexps,
>> this example (which uses neither) _will_ rebind the containing block's
`x`:

> >>

> >> [x := 1]

>
[Chris Barker]
> This reinforces my point that it’s not just about comprehensions,

I agree, it's not at all - and I'm amazed at the over-the-top passion that
minor issues of scope in comprehensions have ... inspired.  It's the tip of
the tail of the dog.

> but rather that the local namespace can be altered anywhere
> an expression is used  — which is everywhere.

Yes, everywhere.  But what of it?  Have you read the PEP?  The examples are
all simple and straightforward and "local".  My example above was wholly
contrived to make a specific point, and I expect we'll _never_ see that
line in real code.

>

> That trivial example is unsurprising, but as soon as your
> line of code gets a bit longer, it could be far more hidden.

It's not possible to prevent people from writing horrible code, and I'm
hard pressed to think of _any_ programming feature that can't be so
abused.  From
ridiculouslyLongVariableNamesWhoseVerbostiySeemsToBeAGoalInItself. massive
overuse of globals, insanely deep nesting, horridly redundant
parenthesization, functions with 20 undocumented arguments, creating
Byzantine class structures spread over a directory full of modules to
implement a concept that _could_ have been done faster and better with a
list, ...

So on a scale of 1 ("wake me up when it's over") to 100 ("OMG!  It's the
end of the world!!!"), "but it can be horridly abused" rates about a 2 on
my weighting scale.  Do we really think so little of our fellow
Pythoneers?  Key point:  absolutely nobody has expressed a fear that they
_themself_ will abuse assignment expressions.  It's always some seemingly
existential dread that someone else will ;-)


> I’m not saying it’s not worth it, but it a more significant

> > complication than simply adding a new feature like augmented

> > assignment or terniary expressions, where the effect is seen only

> > where it is used.

Which is good to keep in mind when using a feature like this.  Python is an
imperative language, and side effects are rampant.  Controlling them is
important.


> A key problem with thinking about this is that we can scan existing
> code to find places where this would improve the code, and decide if
> those use-cases would cause confusion.

I went through that exercise for the PEP's Appendix A.  I assume you
haven't read it.  I found many places where assignment expressions would
make for a small improvement, and got surprised by concluding it was really
the multitude of tiny, extremely-local improvements that "added up" to the
real win overall, not the much rarer cases where assignment expressions
really shine (such as in collapsing chains of semantically misleading
ever-increasing indentation in long
assign/f/else/assign/if/else/assign/if/else ...structures).  I also gave
examples of places where, despite being "small and local" changes, using
assignment expressions appeared to be a _bad_ idea.


> But we really can’t anticipate all the places where it might get used

> > perhaps inappropriately) that would cause confusion. We can hope that
> people won’t tend to do that, but who knows?

Having spent considerable time on it myself (see just above), I do not
assume that other Pythonistas are incapable of reaching sane conclusions
too ;-)


> Example: in a function argument:

> >

> > result = call_a_func(arg1, arg2, kwarg1=x, kwarg2=x:=2*y)

The PEP already calls that one a SyntaxError.  I can't imagine why a sane
programmer would want to do that, but if they really must the PEP _will_
allow it if they parenthesize the assignment expression in this context (so
"kwarg2=(x:=2*y)" instead.


> Sure, there are always ways to write bad code, and most people

> > wouldn’t do that, but someone, somewhere, that thinks shorter code is

> > better code might well do it. Or something like it.

Someone will!  No doubt about it.  But what of it?  If someone is
programming for their own amusement, why should I care?  If they're working
with a group, bad practice should be discouraged by the group's coding
standards and enforced by the group's code review process.  For this
feature, and all others.

"Consenting adults" is a key Python principle too.  And I believe in it,
despite that I wrote tabnanny.py ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker - NOAA Federal via Python-Dev]

> On Jun 28, 2018, at 8:21 PM, Tim Peters  wrote:

Seems it’s all been said, and Tim’s latest response made an excellent
case for consistency.

But:

> Regardless of how assignment expressions work in listcomps and genexps, this 
> example (which uses neither) _will_ rebind the containing block's `x`:
>
> [x := 1]

This reinforces my point that it’s not just about comprehensions, but
rather that the local namespace can be altered anywhere an expression
is used  — which is everywhere.

That trivial example is unsurprising, but as soon as your line of code
gets a bit longer, it could be far more hidden.

I’m not saying it’s not worth it, but it a more significant
complication than simply adding a new feature like augmented
assignment or terniary expressions, where the effect is seen only
where it is used.

A key problem with thinking about this is that we can scan existing
code to find places where this would improve the code, and decide if
those use-cases would cause confusion.

But we really can’t anticipate all the places where it might get used
(perhaps inappropriately) that would cause confusion. We can hope that
people won’t tend to do that, but who knows?

Example: in a function argument:

result = call_a_func(arg1, arg2, kwarg1=x, kwarg2=x:=2*y)

Sure, there are always ways to write bad code, and most people
wouldn’t do that, but someone, somewhere, that thinks shorter code is
better code might well do it. Or something like it.

After all, expressions can be virtually anywhere in your code.

Is this a real risk? Maybe not, but it is a complication.

-CHB
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Terry Reedy]

On 6/28/2018 11:21 PM, Tim Peters wrote:

[somewhere below] this is the last time I'm going to repeat it all again ;-)

For me, this is your most convincing exposition and summary of why the 
proposal is at least ok.  Thank you.




[Chris]
 > yes, it was a contrived example, but the simplest one I could think 
of off

 > the top of my head that re-bound a name in the loop -- which was what I
 > thought was the entire point of this discussion?

But why off the top of your head?  There are literally hundreds & 
hundreds of prior messages about this PEP, not to mention that you could 
also find examples in the PEP.  Why make up a senseless example?


 > If we think hardly anyone is ever going to do that -- then I guess it 
doesn't matter

 > how it's handled.

So look at real examples.  One that's been repeated at least a hundred 
times wants a local to "leak into" a listcomp:


total = 0
cumsums = [total ::= total + value for value in data]

As an educator, how are you going to explain that blowing up with 
UnboundLocalError instead?  Do you currently teach that comprehensions 
and genexps are implemented via invisible magically generated lexically 
nested functions?  If not, you're going to have to start for people to 
even begin to make sense of UnboundLocalError if `total` _doesn't_ "leak 
into" that example.  My belief is that just about everyone who doesn't 
know "too much" about the current implementation will be astonished & 
baffled if that example doesn't "just work".


In other cases it's desired that targets "leak out":

while any(n % (divisor := p) == 0 for p in small_primes):
     n //= divisor

And in still other cases no leaking (neither in nor out) is desired.

Same as `for` targets in that way,. but in the opposite direction:  they 
don't leak and there's no way to make them leak, not even when that's 
wanted.  Which _is_ wanted in the last example above, which would be 
clearer still written as:


while any(n % p == 0 for p in small_primes):
     n //= p

But that ship has sailed.


 > ...
 > And "nonlocal" is not used that often, and when it is it's for 
careful closure

 > trickery -- I'm guessing := will be far more common.

My guess (recorded in the PEP's Appendix A) is that assignment 
expressions _overall_ will be used more often than ternary `if` but 
significantly less often than augmented assignment.  I expect their use 
in genexps and comprehensions will be minimal.  There are real use cases 
for them, but the vast majority of genexps and comprehensions apparently 
have no use for them at all.



 > And, of course, when a newbie encounters it, they can google it and 
see what
 > it means -- far different that seeing a := in a comprehension and 
understanding

 > (by osmosis??) that it might make changes in the local scope.

Which relates to the above:  how do you teach these things?  The idea 
that "a newbie" even _suspects_ that genexps and listcomps have 
something to do with lexically nested scopes and invisible nested 
functions strikes me as hilarious ;-)


Regardless of how assignment expressions work in listcomps and genexps, 
this example (which uses neither) _will_ rebind the containing block's `x`:


[x := 1]

How then are you going to explain that this seemingly trivial variation 
_doesn't_?


[x := 1 for ignore in "a"]

For all the world they both appear to be binding `x` in the code block 
containing the brackets.  So let them.


Even worse,

[x for ignore in range(x := 1)]

will rebind `x` in the containing block _regardless_ of how assignment 
expression targets are treated in "most of" a comprehension, because the 
expression defining the iterable of the outermost "for" _is_ evaluated 
in the containing block (it is _not_ evaluated in the scope of the 
synthetic function).


That's not a special case for targets if they all "leak", but is if they 
don't.



 > And  I don't think you can even do that with generator expressions 
now -- as

 > they can only contain expressions.

Expressions can invoke arbitrary functions, which in turn can do 
anything whatsoever.


 > Which is my point -- this would allow the local namespace to be 
manipulated

 > in places it never could before.

As above, not true.  However, it would make it _easier_ to write 
senseless code mucking with the local namespace - if that's what you 
want to do.



 > Maybe it's only comprehensions, and maybe it'll be rare to have a 
confusing
 > version of those, so it'll be no big deal, but this thread started 
talking about

 > educators' take on this -- and as an educator, I think this really does
 > complicate the language.

I'll grant that it certainly doesn't simplify the language ;-)


 > Python got much of it's "fame" by being "executable pseudo code" -- 
its been
 > moving farther and farther away from those roots. That's generally a 
good thing,
 > as we've gain expressiveness in exchangel, but we shouldn't pretend 
it isn't

 > happening, or that this proposal doesn't contribute to that 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Chris]
> yes, it was a contrived example, but the simplest one I could think of off
> the top of my head that re-bound a name in the loop -- which was what I
> thought was the entire point of this discussion?

But why off the top of your head?  There are literally hundreds & hundreds
of prior messages about this PEP, not to mention that you could also find
examples in the PEP.  Why make up a senseless example?

> If we think hardly anyone is ever going to do that -- then I guess it
doesn't matter
> how it's handled.

So look at real examples.  One that's been repeated at least a hundred
times wants a local to "leak into" a listcomp:

total = 0
cumsums = [total ::= total + value for value in data]

As an educator, how are you going to explain that blowing up with
UnboundLocalError instead?  Do you currently teach that comprehensions and
genexps are implemented via invisible magically generated lexically nested
functions?  If not, you're going to have to start for people to even begin
to make sense of UnboundLocalError if `total` _doesn't_ "leak into" that
example.  My belief is that just about everyone who doesn't know "too much"
about the current implementation will be astonished & baffled if that
example doesn't "just work".

In other cases it's desired that targets "leak out":

while any(n % (divisor := p) == 0 for p in small_primes):
n //= divisor

And in still other cases no leaking (neither in nor out) is desired.

Same as `for` targets in that way,. but in the opposite direction:  they
don't leak and there's no way to make them leak, not even when that's
wanted.  Which _is_ wanted in the last example above, which would be
clearer still written as:

while any(n % p == 0 for p in small_primes):
n //= p

But that ship has sailed.


> ...
> And "nonlocal" is not used that often, and when it is it's for careful
closure
> trickery -- I'm guessing := will be far more common.

My guess (recorded in the PEP's Appendix A) is that assignment expressions
_overall_ will be used more often than ternary `if` but significantly less
often than augmented assignment.  I expect their use in genexps and
comprehensions will be minimal.  There are real use cases for them, but the
vast majority of genexps and comprehensions apparently have no use for them
at all.


> And, of course, when a newbie encounters it, they can google it and see
what
> it means -- far different that seeing a := in a comprehension and
understanding
> (by osmosis??) that it might make changes in the local scope.

Which relates to the above:  how do you teach these things?  The idea that
"a newbie" even _suspects_ that genexps and listcomps have something to do
with lexically nested scopes and invisible nested functions strikes me as
hilarious ;-)

Regardless of how assignment expressions work in listcomps and genexps,
this example (which uses neither) _will_ rebind the containing block's `x`:

[x := 1]

How then are you going to explain that this seemingly trivial variation
_doesn't_?

[x := 1 for ignore in "a"]

For all the world they both appear to be binding `x` in the code block
containing the brackets.  So let them.

Even worse,

[x for ignore in range(x := 1)]

will rebind `x` in the containing block _regardless_ of how assignment
expression targets are treated in "most of" a comprehension, because the
expression defining the iterable of the outermost "for" _is_ evaluated in
the containing block (it is _not_ evaluated in the scope of the synthetic
function).

That's not a special case for targets if they all "leak", but is if they
don't.


> And  I don't think you can even do that with generator expressions now --
as
> they can only contain expressions.

Expressions can invoke arbitrary functions, which in turn can do anything
whatsoever.

> Which is my point -- this would allow the local namespace to be
manipulated
> in places it never could before.

As above, not true.  However, it would make it _easier_ to write senseless
code mucking with the local namespace - if that's what you want to do.


> Maybe it's only comprehensions, and maybe it'll be rare to have a
confusing
> version of those, so it'll be no big deal, but this thread started
talking about
> educators' take on this -- and as an educator, I think this really does
> complicate the language.

I'll grant that it certainly doesn't simplify the language ;-)


> Python got much of it's "fame" by being "executable pseudo code" -- its
been
> moving farther and farther away from those roots. That's generally a good
thing,
> as we've gain expressiveness in exchangel, but we shouldn't pretend it
isn't
> happening, or that this proposal doesn't contribute to that trend.

I didn't say a word about that one way or the other.  I mostly agree, but
at the start Guido was aiming to fill a niche between shell scripting
languages and C.  It was a very "clean" language from the start, but not
aimed at beginners.  Thanks to his experience working on ABC, it carried
over some key ideas that 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Baptiste Carvello wrote:

x=0; [x:=x+i for i in range(5)]


what would be a
non-cryptic alternative to the above example?


Personally I wouldn't insist on trying to do it with a
comprehension at all, but if forced to come up with a
readable syntax for that, it would probably be something
like

   [x for i in range(5) letting x = x + 1 given x = 0]

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Thu, Jun 28, 2018 at 9:28 AM, Tim Peters  wrote:

> >>> g = (x:=i for i in range(3))
> Common or not, I have no idea why anyone would write a genexp like the one
> you gave, except to contrive an example of silly behavior exhibited by
> silly code ;-)
>

yes, it was a contrived example, but the simplest one I could think of off
the top of my head that re-bound a name in the loop -- which was what I
thought was the entire point of this discussion?

If we think hardly anyone is ever going to do that -- then I guess it
doesn't matter how it's handled.

> So while this is technically the same as the comprehension, it is not
> > the same as a generator function which does get its own scope.
>
> It is the same as a generator function with appropriate scope declarations
> - a generator expression is, after all, implemented _by_ a nested generator
> function.  You can write a workalike to your code above today, but nobody
> worries about that because nobody does that ;-)
>
> def f():
> def bashx(outermost):
> nonlocal x
> for i in outermost:
> x = i
> yield i
>

but here the keyword "nonlocal" is used -- you are clearly declaring that
you are messing with a nonlocal name here -- that is a lot more obvious
than simply using a :=

And "nonlocal" is not used that often, and when it is it's for careful
closure trickery -- I'm guessing := will be far more common. And, of
course, when a newbie encounters it, they can google it and see what it
means -- far different that seeing a := in a comprehension and
understanding (by osmosis??) that it might make changes in the local scope.

And  I don't think you can even do that with generator expressions now --
as they can only contain expressions. Which is my point -- this would allow
the local namespace to be manipulated in places it never could before.

Maybe it's only comprehensions, and maybe it'll be rare to have a confusing
version of those, so it'll be no big deal, but this thread started talking
about educators' take on this -- and as an educator, I think this really
does complicate the language.

Python got much of it's "fame" by being "executable pseudo code" -- its
been moving farther and farther away from those roots. That's generally a
good thing, as we've gain expressiveness in exchangel, but we shouldn't
pretend it isn't happening, or that this proposal doesn't contribute to
that trend.

Did adding ternary `if` (truepart if expression else falsepart) complicate
> the language significantly?
>

I don't think so -- no. For two reasons:

1) the final chosen form is kind of verbose, but therefor more like
"executable pseudo code" :-) As apposed to the C version, for instance.

2) it added one new construct, that if, when someone sees it for the first
(or twenty fifth) time and doesn't understand it, they can look it up, and
find out. and it only effects that line of code.

So adding ANYTHING does complicate the language, by simply making it a bit
larger, but some things are far more complicating than others.

Python has rarely expanded the number of expression forms, but whenever it
> has the sky didn't actually fall despite earnest warnings that disaster was
> inevitable ;-)
>

Well, I've been surprised by what confused students before, and I will
again. But I dont hink there is any doubt that Python 3.7 is a notably
harder to learn that Python 1.5 was...


> > Maybe it’s just me, but re-binding a name seems like a whole new
> > category of side effect.
>
> With no trickery at all, you've always been able to rebind attributes, and
> mutate containers, in comprehensions and genexps.  Because `for` targets
> aren't limited to plain names; e.g.,
>
> g = (x+y for object.attribute, a[i][j] in zip(range(3), range(3)))
>

sure, but you are explicitly using the names "object" and "a" here -- so
while side effects in comprehension are discouraged, it's not really a
surprised that namespaces specifically named are changed.

and this:

In [*55*]: x = 0


In [*56*]: [x *for* x *in* range(3)]

Out[*56*]: [0, 1, 2]


In [*57*]: x

Out[*57*]: 0

doesn't change x in the local scope -- if that was a good idea, why is a
good idea to have := in a comprehension effect the local scope??

But maybe it is just me.


> And as in my goofy code above, mucking with binding of plain names is also
> possible today.  Indeed, straightforward if that's what you _want_ to do.
> But nobody does.
>
> It's just not one of Python's goals to make it impossible to write useless
> code ;-)
>

 I suppose we need to go back and look at the "real" examples of where/how
folks think they'll use := in comprehensions, and see how confusing it may
be.

One of these conversations was started with an example something like this:

[(f(x), g(f(x))) for x in an_iterable]

The OP didn't like having to call f() twice. So that would become:

[ (temp:=f(x), g(temp)) for x in an_iterable]

so now the question is: should "temp" be created / changed 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Terry Reedy]

On 6/28/2018 8:05 AM, Baptiste Carvello wrote:

Le 28/06/2018 à 01:31, Greg Ewing a écrit :

Well, I remain profoundly unconvinced that writing comprehensions
with side effects is ever a good idea, and Tim's examples did
nothing to change that.


Comprehensions with side effects feel scary indeed. But I could see
myself using some variant of the "cumsum" example (for scientific work
at the command prompt):


x=0; [x:=x+i for i in range(5)]


Creating an unneeded list with a comprehension purely for side effects 
is considered a bad idea by many.


x = 0
for i in range(5): x += i


Here the side effects are irrelevant, the "x" variable won't be reused.


If we ignore the side effect on x, the above is equivalent to 'pass' ;-)

Perhaps you meant

x = 0
cum = [x:=x+i for i in range(5)]

which is equivalent to

x, cum = 0, []
for i in range(5): x += i; cum.append(x)


But it needs to be initialized at the start of the comprehension.

I would happily get rid of the side-effects, but then what would be a
non-cryptic alternative to the above example?


The above as likely intended can also be written

import itertools as it
cum = list(it.accumulate(range(5)))

We have two good existing alternatives to the proposed innovation.

--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Chris Barker]
>>> So what about:
>>>
>>> l = [x:=i for i in range(3)]
>>>
>>> vs
>>>
>>> g = (x:=i for i in range(3))
>>>
>>> Is there any way to keep these consistent if the "x" is in the regular
local scope?

[Tim]
>> I'm not clear on what the question is.  The list comprehension would
>> bind ` l ` to [0, 1, 2] and leave the local `x` bound to 2.  The second
>> example binds `g` to a generator object, which just sits there
>> unexecuted.  That has nothing to do with the PEP, though.
>>
>> If you go on to do, e.g.,
>>
>> l = list(g)
>>
>> then, same as the listcomp, `l` will be bound to [0, 1, 2] and the local
`x` will
>> be left bound to 2.

[Chris]
> OK, it has been said that the priority is that
>
> list(a_gen_expression)
>
> Behave the same as
>
> [the_same_expression]

That's certainly desirable.


> So we’re good there. And maybe it’s  correct that leaving the running
> of the gen_exp ‘till later is pretty uncommon, particularly for newbies,

Common or not, I have no idea why anyone would write a genexp like the one
you gave, except to contrive an example of silly behavior exhibited by
silly code ;-)

It's really not interesting to me to make up code as goofy as you can
conceive of - the interesting questions are about plausible code (including
plausible coding errors).

>  but:
>
> If the execution of the gen_exp is put off, it really confuses things
> — that name being changed would happen at some arbitrary tone, and at
> least in theory, the gen_exp could be passed off to somewhere else in
> the code, and be run or not run completely remotely from where the
> name is used.

Sure.

> So while this is technically the same as the comprehension, it is not
> the same as a generator function which does get its own scope.

It is the same as a generator function with appropriate scope declarations
- a generator expression is, after all, implemented _by_ a nested generator
function.  You can write a workalike to your code above today, but nobody
worries about that because nobody does that ;-)

def f():
def bashx(outermost):
nonlocal x
for i in outermost:
x = i
yield i

x = 12
g = bashx(range(3))
print("x before", x)
L = list(g)
print("L", L)
print("x after", x)

Then calling `f()` prints:

x before 12
L [0, 1, 2]
x after 2

> And we should be clear how it will work — after all, in py2, the
> handling of the looping name was handled differently in gen_exp vs
> comprehensions.

The PEP specifies the semantics.  If it's accepted, that will be folded
into the docs.

> So I think a local scope for all comprehension-like things would be
> the way to go.
>
> But getting back to the original thread topic — python has a number of
> places that you can only use expressions — adding the ability to bind
> a name in all these places complicates the language significantly.

Did adding ternary `if` (truepart if expression else falsepart) complicate
the language significantly?  Python has rarely expanded the number of
expression forms, but whenever it has the sky didn't actually fall despite
earnest warnings that disaster was inevitable ;-)

>>   Put a body B in a listcomp and any side effects due to executing B

> Maybe it’s just me, but re-binding a name seems like a whole new
> category of side effect.

With no trickery at all, you've always been able to rebind attributes, and
mutate containers, in comprehensions and genexps.  Because `for` targets
aren't limited to plain names; e.g.,

g = (x+y for object.attribute, a[i][j] in zip(range(3), range(3)))

is already "legal", and will stomp all over the complex `for` targets when
executed - there's nothing "local" about them.  But nobody worries about
that because nobody does stuff like that.

And as in my goofy code above, mucking with binding of plain names is also
possible today.  Indeed, straightforward if that's what you _want_ to do.
But nobody does.

It's just not one of Python's goals to make it impossible to write useless
code ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker - NOAA Federal via Python-Dev]

Sent from my iPhone

> > So what about:
> >
> > l = [x:=i for i in range(3)]
> >
> > vs
> >
> > g = (x:=i for i in range(3))
> >
> > Is there any way to keep these consistent if the "x" is in the regular 
> > local scope?
>
> I'm not clear on what the question is.  The list comprehension would bind ` l 
> ` to [0, 1, 2] and leave the local `x` bound to 2.  The second example binds 
> `g` to a generator object, which just sits there unexecuted.  That has 
> nothing to do with the PEP, though.
>
> If you go on to do, e.g.,
>
> l = list(g)
>
> then, same as the listcomp, `l` will be bound to [0, 1, 2] and the local `x` 
> will be left bound to 2.

OK, it has been said that the priority is that

list(a_gen_expression)

Behave the same as

[the_same_expression]

So we’re good there. And maybe it’s  correct that leaving the running
of the gen_exp ‘till later is pretty uncommon, particularly for
newbies, but:

If the execution of the gen_exp is put off, it really confuses things
— that name being changed would happen at some arbitrary tone, and at
least in theory, the gen_exp could be passed off to somewhere else in
the code, and be run or not run completely remotely from where the
name is used.

So while this is technically the same as the comprehension, it is not
the same as a generator function which does get its own scope.

And we should be clear how it will work — after all, in py2, the
handling of the looping name was handled differently in gen_exp vs
comprehensions.

So I think a local scope for all comprehension-like things would be
the way to go.

But getting back to the original thread topic — python has a number of
places that you can only use expressions — adding the ability to bind
a name in all these places complicates the language significantly.

>   Put a body B in a listcomp and any side effects due to executing B

Maybe it’s just me, but re-binding a name seems like a whole new
category of side effect.

-CHB
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Baptiste Carvello]

Le 28/06/2018 à 01:31, Greg Ewing a écrit :
> Well, I remain profoundly unconvinced that writing comprehensions
> with side effects is ever a good idea, and Tim's examples did
> nothing to change that.

Comprehensions with side effects feel scary indeed. But I could see
myself using some variant of the "cumsum" example (for scientific work
at the command prompt):

>>> x=0; [x:=x+i for i in range(5)]

Here the side effects are irrelevant, the "x" variable won't be reused.
But it needs to be initialized at the start of the comprehension.

I would happily get rid of the side-effects, but then what would be a
non-cryptic alternative to the above example?

Baptiste
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Victor Stinner]

✨ Congrats Nick on your 100 emails thread ! ✨ You won a virtual
piece of cake: 

2018-06-22 16:22 GMT+02:00 Nick Coghlan :
> On 22 June 2018 at 02:26, Antoine Pitrou  wrote:
>> Indeed.  But, for a syntax addition such as PEP 572, I think it would be
>> a good idea to ask their opinion to teaching/education specialists.
>>
>> As far as I'm concerned, if teachers and/or education specialists were
>> to say PEP 572 is not a problem, my position would shift from negative
>> towards neutral.
>
> I asked a handful of folks at the Education Summit the next day about it:
>
> * for the basic notion of allowing expression level name binding using
> the "NAME := EXPR" notation, the reactions ranged from mildly negative
> (I read it as only a "-0" rather than a "-1") to outright positive.
> * for the reactions to my description of the currently proposed parent
> local scoping behaviour in comprehensions, I'd use the word
> "horrified", and feel I wasn't overstating the response :)
>
> While I try to account for the fact that I implemented the current
> comprehension semantics for the 3.x series, and am hence biased
> towards considering them the now obvious interpretation, it's also the
> case that generator expressions have worked like nested functions
> since they were introduced in Python 2.4 (more than 13 years ago now),
> and comprehensions have worked the same way as generator expressions
> since Python 3.0 (which has its 10th birthday coming up in December
> this year).
>
> This means that I take any claims that the legacy Python 2.x
> interpretation of comprehension behaviour is intuitively obvious with
> an enormous grain of salt - for the better part of a decade now, every
> tool at a Python 3 user's disposal (the fact that the iteration
> variable is hidden from the current scope, reading the language
> reference [1], printing out locals(), using the dis module, stepping
> through code in a debugger, writing their own tracing function, and
> even observing the quirky interaction with class scopes) will have
> nudged them towards the "it's a hidden nested function" interpretation
> of expected comprehension behaviour.
>
> Acquiring the old mental model for the way comprehensions work pretty
> much requires a developer to have started with Python 2.x themselves
> (perhaps even before comprehensions and lexical closures were part of
> the language), or else have been taught the Python 2 comprehension
> model by someone else - there's nothing in Python 3's behaviour to
> encourage that point of view, and plenty of
> functional-language-inspired documentation to instead encourage folks
> to view comprehensions as tightly encapsulated declarative container
> construction syntax.
>
> I'm currently working on a concept proposal at
> https://github.com/ncoghlan/peps/pull/2 that's much closer to PEP 572
> than any of my previous `given` based suggestions: for already
> declared locals, it devolves to being the same as PEP 572 (except that
> expressions are allowed as top level statements), but for any names
> that haven't been previously introduced, it prohibits assigning to a
> name that doesn't already have a defined scope, and instead relies on
> a new `given` clause on various constructs that allows new target
> declarations to be introduced into the current scope (such that "if
> x:= f():" implies "x" is already defined as a target somewhere else in
> the current scope, while "if x := f() given x:" potentially introduces
> "x" as a new local target the same way a regular assignment statement
> does).
>
> One of the nicer features of the draft proposal is that if all you
> want to do is export the iteration variable from a comprehension, you
> don't need to use an assignment expression at all: you can just append
> "... given global x" or "... given nonlocal x" and export the
> iteration variable directly to the desired outer scope, the same way
> you can in the fully spelled out nested function equivalent.
>
> Cheers,
> Nick.
>
> [1] From 
> https://docs.python.org/3.0/reference/expressions.html#displays-for-lists-sets-and-dictionaries:
> 'Note that the comprehension is executed in a separate scope, so names
> assigned to in the target list don’t “leak” in the enclosing scope.'
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Chris Barker]
> ...
> So what about:
>
> l = [x:=i for i in range(3)]
>
> vs
>
> g = (x:=i for i in range(3))
>
> Is there any way to keep these consistent if the "x" is in the regular
local scope?

I'm not clear on what the question is.  The list comprehension would bind `
l ` to [0, 1, 2] and leave the local `x` bound to 2.  The second example
binds `g` to a generator object, which just sits there unexecuted.  That
has nothing to do with the PEP, though.

If you go on to do, e.g.,

l = list(g)

then, same as the listcomp, `l` will be bound to [0, 1, 2] and the local
`x` will be left bound to 2.

The only real difference is in _when_ the `x:=i for i in range(3)` part
gets executed.  There's no new twist here due to the PEP.  Put a body B in
a listcomp and any side effects due to executing B happen right away, but
put B in a genexp and they don't happen until you force the genexp to yield
results.

For example, do you think these two are "consistent" today?

l = [print(i) for i in range(3)]
g = (print(i) for i in range(3))

?  If so, nothing essential changes by replacing "print(i)" with "x := i" -
in either case the side effects happen when the body is executed.

But if you don't think they're already consistent, then nothing gets less
consistent either ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Barker via Python-Dev]

On Wed, Jun 27, 2018 at 12:01 PM, Eric V. Smith  wrote:

> >>> def test():
> >>>spam = 1
> >>>ham = 2
> >>>vars = [key1+key2 for key1 in locals() for key2 in locals()]
> >>>return vars
> >>>
> >>> Wanna guess what that's gonna return?
>

Guessing aside, messing around with locals() isn't really helpful for the
usual case of common code.

But the real problem I see with all of this the distinction between
generator expressions and comprehensions:

"""
an assignment expression occurring in a list, set or dict comprehension or
in a generator expression (below collectively referred to as
"comprehensions") binds the target in the containing scope, honoring a
nonlocal or global declaration for the target in that scope, if one exists.
For the purpose of this rule the containing scope of a nested comprehension
is the scope that contains the outermost comprehension. A lambda counts as
a containing scope.
"""

It seems everyone agrees that scoping rules should be the same for
generator expressions and comprehensions, which is a good reason for
python3's non-leaking comprehensions:

(py2)

In [*5*]: i = 0


In [*6*]: l = [i *for* i *in* range(3)]


In [*7*]: i

Out[*7*]: 2


In [*8*]: i = 0


In [*9*]: g = (i *for* i *in* range(3))


In [*10*]: i

Out[*10*]: 0


In [*11*]: *for* j *in* g:

...: *pass*

...:


In [*12*]: i

Out[*12*]: 0

so comprehensions and generator expressions behave differently -- not great.

(py3)


In [*4*]: i = 0


In [*5*]: l = [i *for* i *in* range(3)]


In [*6*]: i

Out[*6*]: 0


In [*7*]: g = (i *for* i *in* range(3))


In [*8*]: i

Out[*8*]: 0


In [*9*]: list(g)

Out[*9*]: [0, 1, 2]


In [*10*]: i

Out[*10*]: 0

The loop name doesn't "leak" and comprehensions and generator expressions
are the same this regard -- nice.

So what about:

l = [x:=i for i in range(3)]

vs

g = (x:=i for i in range(3))

Is there any way to keep these consistent if the "x" is in the regular
local scope?

Note that this thread is titled "Informal educator feedback on PEP 572".

As an educator -- this is looking harder an harder to explain to newbies...

Though easier if any assignments made in a "comprehension" don't "leak out".

Which does not mean that we'd need a "proper" new local scope (i.e.
locals() returning something new) -- as long as the common usage was
"intuitive".

>> I'm not singling out Chris here, but these discussions would be easier
> >> to follow and more illuminating if the answers to such puzzles were
> >> presented when they're posed.
>

well, I think the point there was that it wasn't obvious without running
the code -- and that point is made regardless of the answer.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Guido]
> ..
> Given that definition of `__parentlocal`, in first approximation the
> scoping rule proposed by PEP 572 would then be: In comprehensions
> (which in my use in the PEP 572 discussion includes generator
> expressions) the targets of inline assignments are automatically
> endowed with a `__parentlocal` declaration, except inside the
> "outermost iterable" (since that already runs in the parent scope).

If this has to be done ;-) , I suggest removing that last exception.  That
is, "[all] targets of inline assignments in comprehensions are declared
__parentlocal", period, should work fine for (b).  In case one appears in
the outermost iterable of the outermost comprehension, I believe such
declaration is merely semantically redundant, not harmful.

Where "redundant" means someone is so familiar with the implementation that
the scope implications of "already runs in the parent scope" are
immediately clear.  For someone muddy about that, it would be a positive
help to have the intent clarified  by removing the exception.

Plus 99% of the point of "parentlocal" seemed to be to allow mindless
("uniform") by-hand translation of nested comprehensions to nested Python
functions, and an exception for the outermost iterable would work against
that intent.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Angelico]

On Thu, Jun 28, 2018 at 11:29 AM, Ivan Pozdeev via Python-Dev
 wrote:
> On 28.06.2018 2:44, Greg Ewing wrote:
>>
>> Ivan Pozdeev via Python-Dev wrote:
>>>
>>> for me, the primary use case for an assignment expression is to be able
>>> to "catch" a value into a variable in places where I can't put an assignment
>>> statement in, like the infamous `if re.match() is not None'.
>>
>>
>> This seems to be one of only about two uses for assignment
>> expressions that gets regularly brought up. The other is
>> the loop-and-a-half, which is already adequately addressed
>> by iterators.
>>
>> So maybe instead of introducing an out-of-control sledgehammer
>> in the form of ":=", we could think about addressing this
>> particular case.
>>
>> Like maybe adding an "as" clause to if-statements:
>>
>>if pattern.match(s) as m:
>>   do_something_with(m)
>>
>
> I've skimmed for the origins of "as" (which I remember seeing maybe even
> before Py3 was a thing) and found this excellent analysis of modern
> languages which is too a part of the PEP 572 discussion:
> https://mail.python.org/pipermail/python-ideas/2018-May/050920.html
>
> It basically concludes that most recently-created languages do not have
> assignment expressions; they rather allow assignment statement(s?) before
> the tested expression in block statements (only if/while is mentioned. `for'
> is not applicable because its exit condition in Python is always the
> iterable's exhaustion, there's nothing in it that could be used as a
> variable).
>

Now read this response.

https://mail.python.org/pipermail/python-ideas/2018-May/050938.html

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 28.06.2018 2:44, Greg Ewing wrote:

Ivan Pozdeev via Python-Dev wrote:
for me, the primary use case for an assignment expression is to be 
able to "catch" a value into a variable in places where I can't put 
an assignment statement in, like the infamous `if re.match() is not 
None'.


This seems to be one of only about two uses for assignment
expressions that gets regularly brought up. The other is
the loop-and-a-half, which is already adequately addressed
by iterators.

So maybe instead of introducing an out-of-control sledgehammer
in the form of ":=", we could think about addressing this
particular case.

Like maybe adding an "as" clause to if-statements:

   if pattern.match(s) as m:
  do_something_with(m)



I've skimmed for the origins of "as" (which I remember seeing maybe even 
before Py3 was a thing) and found this excellent analysis of modern 
languages which is too a part of the PEP 572 discussion:

https://mail.python.org/pipermail/python-ideas/2018-May/050920.html

It basically concludes that most recently-created languages do not have 
assignment expressions; they rather allow assignment statement(s?) 
before the tested expression in block statements (only if/while is 
mentioned. `for' is not applicable because its exit condition in Python 
is always the iterable's exhaustion, there's nothing in it that could be 
used as a variable).


It, however, doesn't say anything about constructs that are not block 
statements but are equivalent to them, like the ternary operator. (In 
comprehensions, filter conditions are the bits equivalent to if/while 
statements.)


--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Tim]
 > If the parent has a matching parentlocal declaration for the same
> name then the original

> > really refers to the grandparent - and so on.
>

[Greg]

> Ah, I missed that part, sorry -- I withdraw that particular
> objecttion.
>

Good!  I have another reply that crossed in the mail.



> Still, this seems like a major addition (seeing as it comes
> with a new keyword) whose justification is very little more
> than "it makes explaining comprehension scopes easier".
>
> I agree - it has no other sane use case I can see, and "parentlocal" isn't
_needed_ to capture the intended semantics in by-hand translations of
comprehensions.

I don't even think it makes "explaining" easier.  It doesn't eliminate any
corner cases, it just pushes them into the definition of what
"parentllocal" means.

What it would do is make writing synthetic functions "by hand" to implement
comprehensions more uniform, because "parentlocal" would handle the corner
cases by itself instead of making the programmer figure out when and where
they need to type "nonlocal", "global", and/or cruft to establish a name as
local to a block in which the name otherwise does't appear as a binding
target.

But to the extent that doing such translations by hand is meant to be
"educational", it's more educational to learn how to do that stuff yourself.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Ivan Pozdeev via Python-Dev wrote:
for me, the primary use case for an assignment expression 
is to be able to "catch" a value into a variable in places where I can't 
put an assignment statement in, like the infamous `if re.match() is not 
None'.


This seems to be one of only about two uses for assignment
expressions that gets regularly brought up. The other is
the loop-and-a-half, which is already adequately addressed
by iterators.

So maybe instead of introducing an out-of-control sledgehammer
in the form of ":=", we could think about addressing this
particular case.

Like maybe adding an "as" clause to if-statements:

   if pattern.match(s) as m:
  do_something_with(m)

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

> [Nick Coghlan]

> I'm OK with a target scope declaration construct having
> > lexical-scope-dependent behaviour - exactly what "nonlocal NAME" will
> > do depends on both the nature of the current scope,



> [Greg Ewing]
>
Yes, but my point is that having an explicit "parentlocal" scope
> declaration doesn't help to make anything more orthogonal,
> because there's no way it can have *exactly* the same effect
> as a comprehension's implicit parent-local scoping.
>

Sure it can - but I already explained that.

This is the analogy to "nonlocal" Nick is making:  neither "nonlocal" nor
"parentlocal" tell you which scope a declared name _does_ belong to.
Instead they both say "it's not this scope" and specify algorithms you can
follow to determine the scope to which the name does belong.

"parentlocal" isn't an accurate name because the owning scope may not be
the parent block at all, and it may even be a synonym for "global".  I
think "by hand" translations of nested comprehensions into nested functions
are clearer _without_ the "parentlocal" invention.- then you have to be
explicit about what the context requires.  Nick hates that because it isn't
uniform.  I like that because I don't want to pretend a non-uniform thing
is uniform ;-)  The only real use case here is synthesizing nested
functions to implement comprehensions/genexps.


In other words, taking a comprehension and manually expanding
> it into a function with parentlocal declarations wouldn't
> give you something exactly equivalent to the original.
> If that's the purpose of having an explicit parentlocal,
> then it fails at that purpose.
>

You can add (a sufficient number of) parentlocal declarations to get the
precise intended semantics.  Then again, that can also be done today
(without the "parentlocal" invention).


>
> If that's *not* the purpose, then I'm not really sure what
> the purpose is, because I can't think of a situation where
> I'd choose to use parentlocal instead of nonlocal with an
> explicit assignment in the outer scope.
>

For example, if the name is declared "global" in the outer scope, you'll
get a compile-time error if you try to declare it "nonlocal" in the
contained scope.  "parentlocal" adjusts its meaning accordingly, becoming a
synonym for "global" in that specific case.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Tim Peters wrote:
If the parent has a 
matching parentlocal declaration for the same name then the original 
really refers to the grandparent - and so on.


Ah, I missed that part, sorry -- I withdraw that particular
objecttion.

Still, this seems like a major addition (seeing as it comes
with a new keyword) whose justification is very little more
than "it makes explaining comprehension scopes easier".

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 28.06.2018 2:31, Greg Ewing wrote:

Steven D'Aprano wrote:
The *very first* motivating example for this proposal came from a 
comprehension.


I think it is both unfortunate and inevitable that the discussion bogged
down in comprehension-hell.


I think the unfortunateness started when we crossed over from
talking about binding a temporary name for use *within* a
comprehension or expression, to binding a name for use *outside*
the comprehension or expression where it's bound.

I've shown in <05f368c2-3cd2-d7e0-9f91-27afb40d5...@mail.mipt.ru> (27 
Jun 2018 17:07:24 +0300) that assignment expressions are fine in most 
use cases without any changes to scoping whatsoever.


So, as Guido suggested in 
 (26 
Jun 2018 19:36:14 -0700), the scoping matter can be split into a 
separate PEP and discussion.



As long as it's for internal use, whether it's in a comprehension
or not isn't an issue.

Tim Peters has also given a couple of good examples of mathematical 
code that would benefit strongly from this feature.


Going back a few months now, they were the examples that tipped me over


Well, I remain profoundly unconvinced that writing comprehensions
with side effects is ever a good idea, and Tim's examples did
nothing to change that.



--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Ivan Pozdeev via Python-Dev wrote:
This isn't as messy as you make it sound if you remember that the 
outermost iterable is evaluated only once at the start and all the 
others -- each iteration.

Anyone using comprehensions has to know this fact.


That fact alone doesn't imply anthing about the *scopes* in which
those iterators are evaluated, however.

Currently the only situation where the scoping makes a difference
is a generator expression that isn't immediately used, and you can
get a long way into your Python career without ever encountering
that case.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 28.06.2018 2:45, Ivan Pozdeev via Python-Dev wrote:

On 28.06.2018 2:31, Ivan Pozdeev via Python-Dev wrote:

On 28.06.2018 1:42, Steven D'Aprano wrote:
On Wed, Jun 27, 2018 at 05:52:16PM +0300, Ivan Pozdeev via 
Python-Dev wrote:



What this means in practice is that assignments will go to different
scopes depending on *where* they are in the comprehension:

 [ expr   for x in iter1  for y in iter2  if cond ...]
 [ BB for x in AA for y in BB if BB ...]

Assignments in the section marked "AA" will be in the local 
scope;
assignments in the BB sections will be in the sublocal scope. 
That's

not too bad, up to the point you try to assign to the same name in
AA and BB. And then you are likely to get confusing hard to
debug UnboundLocalErrors.

This isn't as messy as you make it sound if you remember that the
outermost iterable is evaluated only once at the start and all the
others -- each iteration.
The question isn't *how often* they are evaluated, or how many loops 
you

have, but *what scope* they are evaluated in. Even in a single loop
comprehension, parts of it are evaluated in the local scope and parts
are evaluated in an implicit sublocal scope.


All expressions inside the comprehension other than the initial 
iterable have access to the loop variables generated by the previous 
parts. So they are necessarily evaluated in the internal scope for 
that to be possible.


Since this is too an essential semantics that one has to know to use 
the construct sensibly, I kinda assumed you could make that 
connection...

E.g.:

[(x*y) for x in range(5) if x%2 for y in range(x,5) if not (x+y)%2]
   A  B  C  D   E

C and D have access to the current x; E and A to both x and y.

This means btw that users cannot rely on there being a single internal 
scope, or a scope at all.
The public guarantee is only the access to the loop variables (and, 
with the PEP, additional variables from assignments), of the current 
iteration, generated by the previous parts.


The expressions in the comprehension just somehow automagically 
determine which of the variables are internal and which are local. How 
they do that is an implementation detail.
And the PEP doesn't need to (and probably shouldn't) make guarantees 
here other than where the variables from expressions are promised to be 
accessible.




The overlap between the two is the trap, if you try to assign to the
same variable in the loop header and then update it in the loop body.

Not to mention the inconsistency that some assignments are accessible
from the surrounding code:

 [expr for a in (x := func(), ...) ]
 print(x)  # works

while the most useful ones, those in the body, will be locked up in an
implicit sublocal scope where they are unreachable from outside of the
comprehension:

 [x := something ...  for a in sequence ]
 print(x)  # fails








--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Nick Coghlan wrote:

I'm OK with a target scope declaration construct having
lexical-scope-dependent behaviour - exactly what "nonlocal NAME" will
do depends on both the nature of the current scope,


Yes, but my point is that having an explicit "parentlocal" scope
declaration doesn't help to make anything more orthogonal,
because there's no way it can have *exactly* the same effect
as a comprehension's implicit parent-local scoping.

In other words, taking a comprehension and manually expanding
it into a function with parentlocal declarations wouldn't
give you something exactly equivalent to the original.
If that's the purpose of having an explicit parentlocal,
then it fails at that purpose.

If that's *not* the purpose, then I'm not really sure what
the purpose is, because I can't think of a situation where
I'd choose to use parentlocal instead of nonlocal with an
explicit assignment in the outer scope. Except maybe for the
class-scope situation, which seems like an extremely obscure
reason to introduce a whole new scoping concept with its
own keyword.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 28.06.2018 2:31, Ivan Pozdeev via Python-Dev wrote:

On 28.06.2018 1:42, Steven D'Aprano wrote:
On Wed, Jun 27, 2018 at 05:52:16PM +0300, Ivan Pozdeev via Python-Dev 
wrote:



What this means in practice is that assignments will go to different
scopes depending on *where* they are in the comprehension:

 [ expr   for x in iter1  for y in iter2  if cond   ...]
 [ BB for x in AA for y in BB if BB ...]

Assignments in the section marked "AA" will be in the local scope;
assignments in the BB sections will be in the sublocal scope. 
That's

not too bad, up to the point you try to assign to the same name in
AA and BB. And then you are likely to get confusing hard to
debug UnboundLocalErrors.

This isn't as messy as you make it sound if you remember that the
outermost iterable is evaluated only once at the start and all the
others -- each iteration.

The question isn't *how often* they are evaluated, or how many loops you
have, but *what scope* they are evaluated in. Even in a single loop
comprehension, parts of it are evaluated in the local scope and parts
are evaluated in an implicit sublocal scope.


All expressions inside the comprehension other than the initial 
iterable have access to the loop variables generated by the previous 
parts. So they are necessarily evaluated in the internal scope for 
that to be possible.


Since this is too an essential semantics that one has to know to use 
the construct sensibly, I kinda assumed you could make that connection...

E.g.:

[(x*y) for x in range(5) if x%2 for y in range(x,5) if not (x+y)%2]
   A  B  C  D   E

C and D have access to the current x; E and A to both x and y.

This means btw that users cannot rely on there being a single internal 
scope, or a scope at all.
The public guarantee is only the access to the loop variables (and, with 
the PEP, additional variables from assignments), of the current 
iteration, generated by the previous parts.




The overlap between the two is the trap, if you try to assign to the
same variable in the loop header and then update it in the loop body.

Not to mention the inconsistency that some assignments are accessible
from the surrounding code:

 [expr for a in (x := func(), ...) ]
 print(x)  # works

while the most useful ones, those in the body, will be locked up in an
implicit sublocal scope where they are unreachable from outside of the
comprehension:

 [x := something ...  for a in sequence ]
 print(x)  # fails






--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 28.06.2018 1:42, Steven D'Aprano wrote:

On Wed, Jun 27, 2018 at 05:52:16PM +0300, Ivan Pozdeev via Python-Dev wrote:


What this means in practice is that assignments will go to different
scopes depending on *where* they are in the comprehension:

 [ expr   for x in iter1  for y in iter2  if cond   ...]
 [ BB for x in AA for y in BB if BB ...]

Assignments in the section marked "AA" will be in the local scope;
assignments in the BB sections will be in the sublocal scope. That's
not too bad, up to the point you try to assign to the same name in
AA and BB. And then you are likely to get confusing hard to
debug UnboundLocalErrors.

This isn't as messy as you make it sound if you remember that the
outermost iterable is evaluated only once at the start and all the
others -- each iteration.

The question isn't *how often* they are evaluated, or how many loops you
have, but *what scope* they are evaluated in. Even in a single loop
comprehension, parts of it are evaluated in the local scope and parts
are evaluated in an implicit sublocal scope.


All expressions inside the comprehension other than the initial iterable 
have access to the loop variables generated by the previous parts. So 
they are necessarily evaluated in the internal scope for that to be 
possible.


Since this is too an essential semantics that one has to know to use the 
construct sensibly, I kinda assumed you could make that connection...

E.g.:

[(x*y) for x in range(5) if x%2 for y in range(x,5) if not (x+y)%2]
   A  B  C  D   E

C and D have access to the current x; E and A to both x and y.



The overlap between the two is the trap, if you try to assign to the
same variable in the loop header and then update it in the loop body.

Not to mention the inconsistency that some assignments are accessible
from the surrounding code:

 [expr for a in (x := func(), ...) ]
 print(x)  # works

while the most useful ones, those in the body, will be locked up in an
implicit sublocal scope where they are unreachable from outside of the
comprehension:

 [x := something ...  for a in sequence ]
 print(x)  # fails




--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Steven D'Aprano wrote:
The *very first* motivating example for this proposal came from a 
comprehension.


I think it is both unfortunate and inevitable that the discussion bogged
down in comprehension-hell.


I think the unfortunateness started when we crossed over from
talking about binding a temporary name for use *within* a
comprehension or expression, to binding a name for use *outside*
the comprehension or expression where it's bound.

As long as it's for internal use, whether it's in a comprehension
or not isn't an issue.

Tim Peters has also given a 
couple of good examples of mathematical code that would benefit strongly 
from this feature.


Going back a few months now, they were the examples that tipped me over


Well, I remain profoundly unconvinced that writing comprehensions
with side effects is ever a good idea, and Tim's examples did
nothing to change that.

--
Greg

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Eric Fahlgren]

On Wed, Jun 27, 2018 at 9:27 AM Paul Moore  wrote:

> From my reading, PEP 572 takes the position that "parent local
> scoping" is what people expect from assignment expressions *in
> comprehensions* and it's useful enough that there is no reason not to
> make that the behaviour. The behaviour isn't generally useful enough
> to be worth exposing as a primitive (it's not even useful enough for
> the PEP to give it an explicit name!) so it's just a special case for
> assignment expressions in comprehensions/generators.
>

​So, my interpretation is that it will behave like this?

x = 2
y = [x := 3 for i in range(1)]
print(x)
3

def f():
x = 4
y = [x := 5 for i in range(1)]
print(x)
f()
5

class C:
x = 6
y = [x := 7 for i in range(1)]
print(x)
C()
6
print(x)
7​
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 05:52:16PM +0300, Ivan Pozdeev via Python-Dev wrote:

> >What this means in practice is that assignments will go to different
> >scopes depending on *where* they are in the comprehension:
> >
> > [ expr   for x in iter1  for y in iter2  if cond   ...]
> > [ BB for x in AA for y in BB if BB ...]
> >
> >Assignments in the section marked "AA" will be in the local scope;
> >assignments in the BB sections will be in the sublocal scope. That's
> >not too bad, up to the point you try to assign to the same name in
> >AA and BB. And then you are likely to get confusing hard to
> >debug UnboundLocalErrors.
> 
> This isn't as messy as you make it sound if you remember that the 
> outermost iterable is evaluated only once at the start and all the 
> others -- each iteration.

The question isn't *how often* they are evaluated, or how many loops you 
have, but *what scope* they are evaluated in. Even in a single loop 
comprehension, parts of it are evaluated in the local scope and parts 
are evaluated in an implicit sublocal scope.

The overlap between the two is the trap, if you try to assign to the 
same variable in the loop header and then update it in the loop body.

Not to mention the inconsistency that some assignments are accessible 
from the surrounding code:

[expr for a in (x := func(), ...) ]
print(x)  # works

while the most useful ones, those in the body, will be locked up in an 
implicit sublocal scope where they are unreachable from outside of the 
comprehension:

[x := something ...  for a in sequence ]
print(x)  # fails


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Guido van Rossum]

So IIUC you are okay with the behavior described by the PEP but you want an
explicit language feature to specify it?

I don't particularly like adding a `parentlocal` statement to the language,
because I don't think it'll be generally useful. (We don't have `goto` in
the language even though it could be used in the formal specification of
`if`, for example. :-)

But as a descriptive mechanism to make the PEP's spec clearer I'm fine with
it. Let's call it `__parentlocal` for now. It would work a bit like
`nonlocal` but also different, since in the normal case (when there's no
matching `nonlocal` in the parent scope) it would make the target a local
in that scope rather than trying to look for a definition of the target
name in surrounding (non-class, non-global) scopes. Also if there's a
matching `global` in the parent scope, `__parentlocal` itself changes its
meaning to `global`. If you want to push a target through several level of
target scopes you can do that by having a `__parentlocal` in each scope
that it should push through (this is needed for nested comprehensions, see
below).

Given that definition of `__parentlocal`, in first approximation the
scoping rule proposed by PEP 572 would then be: In comprehensions (which in
my use in the PEP 572 discussion includes generator expressions) the
targets of inline assignments are automatically endowed with a
`__parentlocal` declaration, except inside the "outermost iterable" (since
that already runs in the parent scope).

There would have to be additional words when comprehensions themselves are
nested (e.g. `[[a for a in range(i)] for i in range(10)]`) since the PEP's
intention is that inline assignments anywhere there end up targeting the
scope containing the outermost comprehension. But this can all be expressed
by adding `__parentlocal` for various variables in various places
(including in the "outermost iterable" of inner comprehensions).

I'd also like to keep the rule prohibiting use of the same name as a
comprehension loop control variable and as an inline assignment target;
this rule would also prohibit shenanigans with nested comprehensions (for
any set of nested comprehensions, any name that's a loop control variable
in any of them cannot be an inline assignment target in any of them). This
would also apply to the "outermost iterable".

Does this help at all, or did I miss something?

--Guido

On Wed, Jun 27, 2018 at 5:27 AM Nick Coghlan  wrote:

> On 26 June 2018 at 02:27, Guido van Rossum  wrote:
> > [This is my one reply in this thread today. I am trying to limit the
> amount
> > of time I spend to avoid another overheated escalation.]
>
> Aye, I'm trying to do the same, and deliberately spending some
> evenings entirely offline is helping with that :)
>
> > On Mon, Jun 25, 2018 at 4:44 AM Nick Coghlan  wrote:
> >>
> >> Right, the proposed blunt solution to "Should I use 'NAME = EXPR' or
> >> 'NAME := EXPR'?" bothers me a bit, but it's the implementation
> >> implications of parent local scoping that I fear will create a
> >> semantic tar pit we can't get out of later.
> >
> > Others have remarked this too, but it really bother me that you are
> focusing
> > so much on the implementation of parent local scoping rather than on the
> > "intuitive" behavior which is super easy to explain -- especially to
> someone
> > who isn't all that familiar (or interested) with the implicit scope
> created
> > for the loop control variable(s). According to Steven (who noticed that
> this
> > is barely mentioned in most tutorials about comprehensions) that is most
> > people, however very few of them read python-dev.
> >
> > It's not that much work for the compiler, since it just needs to do a
> little
> > bit of (new) static analysis and then it can generate the bytecode to
> > manipulate closure(s). The runtime proper doesn't need any new
> > implementation effort. The fact that sometimes a closure must be
> introduced
> > where no explicit initialization exists is irrelevant to the runtime --
> this
> > only affects the static analysis, at runtime it's no different than if
> the
> > explicit initialization was inside `if 0`.
>
> One of the things I prize about Python's current code generator is how
> many of the constructs can be formulated as simple content-and-context
> independent boilerplate removal, which is why parent local scoping (as
> currently defined in PEP 572) bothers me: rather than being a new
> primitive in its own right, the PEP instead makes the notion of "an
> assignment expression in a comprehension or generator expression" a
> construct that can't readily decomposed into lower level building
> blocks the way that both assignment expressions on their own and
> comprehensions and generator expressions on their own can be. Instead,
> completely new language semantics arise from the interaction between
> two otherwise independent features.
>
> Even changes as complicated as PEP 343's with statement, PEP 380's
> yield from, and PEP 492's 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Eric V. Smith]

> On Jun 27, 2018, at 9:49 AM, Steven D'Aprano  wrote:
> 
>> On Wed, Jun 27, 2018 at 08:00:20AM -0400, Eric V. Smith wrote:
>>> On 6/27/2018 7:08 AM, Chris Angelico wrote:
>>> It gets funnier with nested loops. Or scarier. I've lost the ability
>>> to distinguish those two.
>>> 
>>> def test():
>>>spam = 1
>>>ham = 2
>>>vars = [key1+key2 for key1 in locals() for key2 in locals()]
>>>return vars
>>> 
>>> Wanna guess what that's gonna return?
>> 
>> I'm not singling out Chris here, but these discussions would be easier 
>> to follow and more illuminating if the answers to such puzzles were 
>> presented when they're posed.
> 
> You can just copy and paste the function into the interactive 
> interpreter and run it :-)

Not on my phone when I’m riding a bus, I can’t. I’m trying to more or less 
follow the discussion, but the “guess what this will do” aspect of the 
discussion makes it hard.

Eric 

> 
> But where's the fun in that? The point of the exercise is to learn first 
> hand just how complicated it is to try to predict the *current* scope 
> behaviour of comprehensions. Without the ability to perform assignment 
> inside them, aside from the loop variable, we've managed to avoid 
> thinking too much about this until now.
> 
> It also demonstrates the unrealisticness of treating comprehensions as a 
> separate scope -- they're hybrid scope, with parts of the comprehension 
> running in the surrounding local scope, and parts running in an sublocal 
> scope.
> 
> Earlier in this thread, Nick tried to justify the idea that 
> comprehensions run in their own scope, no matter how people think of 
> them -- but that's an over-simplification, as Chris' example above 
> shows. Parts of the comprehension do in fact behave exactly as the naive 
> model would suggest (even if Nick is right that other parts don't).
> 
> As complicated and hairy as the above example is, (1) it is a pretty 
> weird thing to do, so most of us will almost never need to consider it; 
> and (2) backwards compatibility requires that we live with it now (at 
> least unless we introduce a __future__ import).
> 
> If we can't simplify the scope of comprehensions, we can at least 
> simplify the parts that actually matters. What matters are the loop 
> variables (already guaranteed to be sublocal and not "leak" out of the 
> comprehension) and the behaviour of assignment expressions (open to 
> discussion).
> 
> Broadly speaking, there are two positions we can take:
> 
> 1. Let the current implementation of comprehensions as an implicit 
> hidden function drive the functionality; that means we duplicate the 
> hairiness of the locals() behaviour seen above, although it won't be 
> obvious at first glance.
> 
> What this means in practice is that assignments will go to different 
> scopes depending on *where* they are in the comprehension:
> 
>[ expr   for x in iter1  for y in iter2  if cond   ...]
>[ BB for x in AA for y in BB if BB ...]
> 
> Assignments in the section marked "AA" will be in the local scope; 
> assignments in the BB sections will be in the sublocal scope. That's 
> not too bad, up to the point you try to assign to the same name in 
> AA and BB. And then you are likely to get confusing hard to 
> debug UnboundLocalErrors.
> 
> 
> 2. Or we can keep the current behaviour for locals and the loop 
> variables, but we can keep assignment expressions simple by ensuring 
> they always bind to the enclosing scope. Compared to the complexity of 
> the above, we have the relatively straight forward:
> 
>[ AA for x in AA for y in AA if AA ...]
> 
> The loop variables continue to be hidden away in the invisible, implicit 
> comprehension function, where they can't leak out, while explicit 
> assignments to variables (using := or given or however it is spelled) 
> will always go into the surrounding local scope, like they do in every 
> other expression.
> 
> Does it matter that the implementation of this requires an implicit 
> nonlocal declaration for each assignment? No more than it matters that 
> comprehensions themselves require an implicit function.
> 
> And what we get out of this is simpler semantics at the Python level:
> 
> - Unless previous declared global, assignment expressions always bind to 
> the current scope, even if they're inside a comprehension;
> 
> - and we don't have to deal with the oddity that different bits of a 
> comprehension run in different scopes (unless we go out of our way to 
> use locals()); merely using assignment expressions will just work 
> consistently and simply, and loop variables will still be confined to 
> the comprehension as they are now.
> 
> 
> -- 
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Nick Coghlan]

> However, PEP 572 in its current form takes the position "parent local
> scoping is sufficiently useful to make it a required pre-requisite for
> adding assignment expressions, but not useful enough to expose as a
> new scope declaration primitive",
>

Of course the PEP doesn't take that position at all:  it doesn't even
contain the term "parent local scoping".  That's your term, which nobody
else uses unless they're replying to you ;-)

What the PEP does say:

"""
an assignment expression occurring in a list, set or dict comprehension or
in a generator expression (below collectively referred to as
"comprehensions") binds the target in the containing scope, honoring a
nonlocal or global declaration for the target in that scope, if one exists.
For the purpose of this rule the containing scope of a nested comprehension
is the scope that contains the outermost comprehension. A lambda counts as
a containing scope.
"""

It's a small collection of plainly stated rules for specifying the intended
semantics.  If you want to claim that this _is_ "useful enough to expose as
a new scope declaration primitive", it's really on you to present use cases
to justify that claim.  I'd present some for you, but I don't have any (I
don't care that "by hand" conversion of nested comprehensions to workalike
Python nested functions may require a bit of thought to establish the
intended scope of assignment expression target names - all of which is
easily doable without adding any new statements).

I don't _expect_ that other good use cases exist.  The gimmick's purpose is
to make code that visually _appears_ to belong to a block act as if
embedded assignments do occur in that block.  If there's an explicitly
nested function, that fundamental motivation no longer applies.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Tim Peters]

[Nick Coghlan]>

>  actually made those semantics available as an explicit

> "parentlocal NAME" declaration ...:
> >
> > def _list_comp(_outermost_iter):
> > parentlocal item
> > _result = []
> > for x in _outermost_iter:
> > item = x
> > _result.append(x)
> > return _result
> >
> > _expr_result = _list_comp(items)
>

[Greg Ewing]

I'm not sure that's possible. If I understand correctly,
> part of the definition of "parent local" is that "parent"
> refers to the nearest enclosing *non-comprehension* scope,
> to give the expected result for nested comprehensions.
> If that's so, then it's impossible to fully decouple its
> definition from comprehensions.
>
>  Nick's "parentlocal" does refer to the parent, but makes no distinction
between synthesized and user-written functions.  If the parent has a
matching parentlocal declaration for the same name then the original really
refers to the grandparent - and so on.  Ultimately, it resolves to the
closest enclosing scope in which the name is _not_ declared parentlocal.
In that scope, a "nonlocal" or "global" declaration settles it if one
appears, else the name is local to that scope.

So a nested comprehension would declare its assignment expression targets
as parentlocal in its synthesized function, and in all the containing
synthesized functions generated for containing comprehensions.

This appears in some strained ;-) way "natural" only because there is no
explicit way to declare something "local" in Python.  In just about any
other language with closures and nested lexical scopes, comprehensions and
generator expressions would have been implemented via nested functions that
explicitly declared their "for" target names "local". and nothing else.
The only change needed then for PEP 572 (b) semantics would be to declare
assignment expression target names local (if their scope wasn't already
known) in the closest containing non-synthesized block.

None of which really matters.  The real question is which semantics are
desired.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Paul Moore]

On 27 June 2018 at 15:39, Nick Coghlan  wrote:

> However, PEP 572 in its current form takes the position "parent local
> scoping is sufficiently useful to make it a required pre-requisite for
> adding assignment expressions, but not useful enough to expose as a
> new scope declaration primitive", and I've come to the view that it
> really is the "A+B=MAGIC!" aspect of the current proposal that bothers
> me, whereas "A+B implies C for " doesn't bother me
> any more than the implicit non-local references introduced as part of
> the original lexical scoping changes bother me.

>From my reading, PEP 572 takes the position that "parent local
scoping" is what people expect from assignment expressions *in
comprehensions* and it's useful enough that there is no reason not to
make that the behaviour. The behaviour isn't generally useful enough
to be worth exposing as a primitive (it's not even useful enough for
the PEP to give it an explicit name!) so it's just a special case for
assignment expressions in comprehensions/generators.

That seems to me like a classic example of practicality beating purity.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 27.06.2018 16:49, Steven D'Aprano wrote:

On Wed, Jun 27, 2018 at 08:00:20AM -0400, Eric V. Smith wrote:

On 6/27/2018 7:08 AM, Chris Angelico wrote:

It gets funnier with nested loops. Or scarier. I've lost the ability
to distinguish those two.

def test():
 spam = 1
 ham = 2
 vars = [key1+key2 for key1 in locals() for key2 in locals()]
 return vars

Wanna guess what that's gonna return?

I'm not singling out Chris here, but these discussions would be easier
to follow and more illuminating if the answers to such puzzles were
presented when they're posed.

You can just copy and paste the function into the interactive
interpreter and run it :-)

But where's the fun in that? The point of the exercise is to learn first
hand just how complicated it is to try to predict the *current* scope
behaviour of comprehensions. Without the ability to perform assignment
inside them, aside from the loop variable, we've managed to avoid
thinking too much about this until now.

It also demonstrates the unrealisticness of treating comprehensions as a
separate scope -- they're hybrid scope, with parts of the comprehension
running in the surrounding local scope, and parts running in an sublocal
scope.

Earlier in this thread, Nick tried to justify the idea that
comprehensions run in their own scope, no matter how people think of
them -- but that's an over-simplification, as Chris' example above
shows. Parts of the comprehension do in fact behave exactly as the naive
model would suggest (even if Nick is right that other parts don't).

As complicated and hairy as the above example is, (1) it is a pretty
weird thing to do, so most of us will almost never need to consider it;
and (2) backwards compatibility requires that we live with it now (at
least unless we introduce a __future__ import).

If we can't simplify the scope of comprehensions, we can at least
simplify the parts that actually matters. What matters are the loop
variables (already guaranteed to be sublocal and not "leak" out of the
comprehension) and the behaviour of assignment expressions (open to
discussion).

Broadly speaking, there are two positions we can take:

1. Let the current implementation of comprehensions as an implicit
hidden function drive the functionality; that means we duplicate the
hairiness of the locals() behaviour seen above, although it won't be
obvious at first glance.

What this means in practice is that assignments will go to different
scopes depending on *where* they are in the comprehension:

 [ expr   for x in iter1  for y in iter2  if cond   ...]
 [ BB for x in AA for y in BB if BB ...]

Assignments in the section marked "AA" will be in the local scope;
assignments in the BB sections will be in the sublocal scope. That's
not too bad, up to the point you try to assign to the same name in
AA and BB. And then you are likely to get confusing hard to
debug UnboundLocalErrors.


This isn't as messy as you make it sound if you remember that the 
outermost iterable is evaluated only once at the start and all the 
others -- each iteration.

Anyone using comprehensions has to know this fact.
The very readable syntax also makes it rather straightforward (though 
admittedly requiring some hand-tracing) to figure out what is evaluated 
after what.




2. Or we can keep the current behaviour for locals and the loop
variables, but we can keep assignment expressions simple by ensuring
they always bind to the enclosing scope. Compared to the complexity of
the above, we have the relatively straight forward:

 [ AA for x in AA for y in AA if AA ...]

The loop variables continue to be hidden away in the invisible, implicit
comprehension function, where they can't leak out, while explicit
assignments to variables (using := or given or however it is spelled)
will always go into the surrounding local scope, like they do in every
other expression.

Does it matter that the implementation of this requires an implicit
nonlocal declaration for each assignment? No more than it matters that
comprehensions themselves require an implicit function.

And what we get out of this is simpler semantics at the Python level:

- Unless previous declared global, assignment expressions always bind to
the current scope, even if they're inside a comprehension;

- and we don't have to deal with the oddity that different bits of a
comprehension run in different scopes (unless we go out of our way to
use locals()); merely using assignment expressions will just work
consistently and simply, and loop variables will still be confined to
the comprehension as they are now.




--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 27 June 2018 at 23:38, Greg Ewing  wrote:
> Nick Coghlan wrote:
>>
>> actually made those semantics available as an explicit
>> "parentlocal NAME" declaration ...:
>>
>> def _list_comp(_outermost_iter):
>> parentlocal item
>> _result = []
>> for x in _outermost_iter:
>> item = x
>> _result.append(x)
>> return _result
>>
>> _expr_result = _list_comp(items)
>
>
> I'm not sure that's possible. If I understand correctly,
> part of the definition of "parent local" is that "parent"
> refers to the nearest enclosing *non-comprehension* scope,
> to give the expected result for nested comprehensions.
> If that's so, then it's impossible to fully decouple its
> definition from comprehensions.

I'm OK with a target scope declaration construct having
lexical-scope-dependent behaviour - exactly what "nonlocal NAME" will
do depends on both the nature of the current scope, and on which names
are declared as local in which outer scopes, and that's also
implicitly the case for all name lookups.

However, PEP 572 in its current form takes the position "parent local
scoping is sufficiently useful to make it a required pre-requisite for
adding assignment expressions, but not useful enough to expose as a
new scope declaration primitive", and I've come to the view that it
really is the "A+B=MAGIC!" aspect of the current proposal that bothers
me, whereas "A+B implies C for " doesn't bother me
any more than the implicit non-local references introduced as part of
the original lexical scoping changes bother me.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 27.06.2018 16:25, Greg Ewing wrote:

Ivan Pozdeev via Python-Dev wrote:
Using this assigned result elsewhere in the same expression (akin to 
regex backreferences) is not a part of the basic idea actually.


If that's true, then the proposal has mutated into something
that has *no* overlap whatsoever with the use case that started
this whole discussion,


I don't know what and where "started" it (AFAIK the idea has been around 
for years) but for me, the primary use case for an assignment expression 
is to be able to "catch" a value into a variable in places where I can't 
put an assignment statement in, like the infamous `if re.match() is not 
None'.



which was about binding a temporary
variable in a comprehension, for use *within* the comprehension.


Then I can't understand all the current fuss about scoping.
AFAICS, it's already like I described in 
https://mail.python.org/pipermail/python-dev/2018-June/154067.html :
the outermost iterable is evaluated in the local scope while others in 
the internal one:


In [13]: [(l,i) for l in list(locals())[:5] for i in locals()]
Out[13]:
[('__name__', 'l'),
 ('__name__', '.0'),
 ('__builtin__', 'l'),
 ('__builtin__', '.0'),
 ('__builtin__', 'i'),
 ('__builtins__', 'l'),
 ('__builtins__', '.0'),
 ('__builtins__', 'i'),
 ('_ih', 'l'),
 ('_ih', '.0'),
 ('_ih', 'i'),
 ('_oh', 'l'),
 ('_oh', '.0'),
 ('_oh', 'i')]

(note that `i' is bound after the first evaluation of internal 
`locals()' btw, as to be expected)


If the "temporary variables" are for use inside the comprehension only, 
the assignment expression needs to bind in the current scope like the 
regular assignment statement, no changes are needed!


It depends on the evaluation order (and whether something is 
evaluated at all),


Which to my mind is yet another reason not to like ":=".



--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 03:41:23PM +0200, Antoine Pitrou wrote:
> 
> Why is this discussion talking about comprehensions at all?
> Is there a decent use case for using assignments in comprehensions (as
> opposed to language lawyering or deliberate obfuscation)?

Yes. The *very first* motivating example for this proposal came from a 
comprehension.

I think it is both unfortunate and inevitable that the discussion bogged
down in comprehension-hell. Unfortunate because I don't think that the 
most compelling use-cases involve comprehensions at all. But inevitable 
because *comprehensions are the hard case*, thanks to the (justifiable!) 
decision to implement them as implicit hidden functions.

In my opinion, the really two BIG wins for assignment expressions are 
while loops and cascades of if... blocks. Tim Peters has also given a 
couple of good examples of mathematical code that would benefit strongly 
from this feature.

Going back a few months now, they were the examples that tipped me over 
from the opinion

"Oh, just re-write the comprehension as a loop"

to the opinion

"You know, I think this feature actually is useful... and
as a bonus, you can keep using the comprehension"

But that requires that we get the comprehension scoping right. Not just 
leave it as an unspecified implementation detail.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 08:00:20AM -0400, Eric V. Smith wrote:
> On 6/27/2018 7:08 AM, Chris Angelico wrote:
> >It gets funnier with nested loops. Or scarier. I've lost the ability
> >to distinguish those two.
> >
> >def test():
> > spam = 1
> > ham = 2
> > vars = [key1+key2 for key1 in locals() for key2 in locals()]
> > return vars
> >
> >Wanna guess what that's gonna return?
> 
> I'm not singling out Chris here, but these discussions would be easier 
> to follow and more illuminating if the answers to such puzzles were 
> presented when they're posed.

You can just copy and paste the function into the interactive 
interpreter and run it :-)

But where's the fun in that? The point of the exercise is to learn first 
hand just how complicated it is to try to predict the *current* scope 
behaviour of comprehensions. Without the ability to perform assignment 
inside them, aside from the loop variable, we've managed to avoid 
thinking too much about this until now.

It also demonstrates the unrealisticness of treating comprehensions as a 
separate scope -- they're hybrid scope, with parts of the comprehension 
running in the surrounding local scope, and parts running in an sublocal 
scope.

Earlier in this thread, Nick tried to justify the idea that 
comprehensions run in their own scope, no matter how people think of 
them -- but that's an over-simplification, as Chris' example above 
shows. Parts of the comprehension do in fact behave exactly as the naive 
model would suggest (even if Nick is right that other parts don't).

As complicated and hairy as the above example is, (1) it is a pretty 
weird thing to do, so most of us will almost never need to consider it; 
and (2) backwards compatibility requires that we live with it now (at 
least unless we introduce a __future__ import).

If we can't simplify the scope of comprehensions, we can at least 
simplify the parts that actually matters. What matters are the loop 
variables (already guaranteed to be sublocal and not "leak" out of the 
comprehension) and the behaviour of assignment expressions (open to 
discussion).

Broadly speaking, there are two positions we can take:

1. Let the current implementation of comprehensions as an implicit 
hidden function drive the functionality; that means we duplicate the 
hairiness of the locals() behaviour seen above, although it won't be 
obvious at first glance.

What this means in practice is that assignments will go to different 
scopes depending on *where* they are in the comprehension:

[ expr   for x in iter1  for y in iter2  if cond   ...]
[ BB for x in AA for y in BB if BB ...]

Assignments in the section marked "AA" will be in the local scope; 
assignments in the BB sections will be in the sublocal scope. That's 
not too bad, up to the point you try to assign to the same name in 
AA and BB. And then you are likely to get confusing hard to 
debug UnboundLocalErrors.


2. Or we can keep the current behaviour for locals and the loop 
variables, but we can keep assignment expressions simple by ensuring 
they always bind to the enclosing scope. Compared to the complexity of 
the above, we have the relatively straight forward:

[ AA for x in AA for y in AA if AA ...]

The loop variables continue to be hidden away in the invisible, implicit 
comprehension function, where they can't leak out, while explicit 
assignments to variables (using := or given or however it is spelled) 
will always go into the surrounding local scope, like they do in every 
other expression.

Does it matter that the implementation of this requires an implicit 
nonlocal declaration for each assignment? No more than it matters that 
comprehensions themselves require an implicit function.

And what we get out of this is simpler semantics at the Python level:

- Unless previous declared global, assignment expressions always bind to 
the current scope, even if they're inside a comprehension;

- and we don't have to deal with the oddity that different bits of a 
comprehension run in different scopes (unless we go out of our way to 
use locals()); merely using assignment expressions will just work 
consistently and simply, and loop variables will still be confined to 
the comprehension as they are now.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Antoine Pitrou]

Why is this discussion talking about comprehensions at all?
Is there a decent use case for using assignments in comprehensions (as
opposed to language lawyering or deliberate obfuscation)?

Regards

Antoine.


On Thu, 28 Jun 2018 01:25:14 +1200
Greg Ewing  wrote:
> Ivan Pozdeev via Python-Dev wrote:
> > Using this assigned result elsewhere in the same expression (akin to 
> > regex backreferences) is not a part of the basic idea actually.  
> 
> If that's true, then the proposal has mutated into something
> that has *no* overlap whatsoever with the use case that started
> this whole discussion, which was about binding a temporary
> variable in a comprehension, for use *within* the comprehension.
> 
> > It depends on the evaluation order (and whether something is evaluated 
> > at all),  
> 
> Which to my mind is yet another reason not to like ":=".
> 



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Nick Coghlan wrote:

actually made those semantics available as an explicit
"parentlocal NAME" declaration ...:

def _list_comp(_outermost_iter):
parentlocal item
_result = []
for x in _outermost_iter:
item = x
_result.append(x)
return _result

_expr_result = _list_comp(items)


I'm not sure that's possible. If I understand correctly,
part of the definition of "parent local" is that "parent"
refers to the nearest enclosing *non-comprehension* scope,
to give the expected result for nested comprehensions.
If that's so, then it's impossible to fully decouple its
definition from comprehensions.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Ivan Pozdeev via Python-Dev wrote:
Using this assigned result elsewhere in the same expression (akin to 
regex backreferences) is not a part of the basic idea actually.


If that's true, then the proposal has mutated into something
that has *no* overlap whatsoever with the use case that started
this whole discussion, which was about binding a temporary
variable in a comprehension, for use *within* the comprehension.

It depends on the evaluation order (and whether something is evaluated 
at all),


Which to my mind is yet another reason not to like ":=".

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 27.06.2018 5:36, Guido van Rossum wrote:

[This is my one response today]

On Mon, Jun 25, 2018 at 12:40 PM Terry Reedy > wrote:


On 6/24/2018 7:25 PM, Guido van Rossum wrote:
> I'd wager that the people who might be most horrified about it

the (b) scoping rule change

> would be people who feel strongly that the change to the
> comprehension scope rules in Python 3 is a big improvement,

I might not be one of those 'most horrified' by (b), but I
increasingly
don't like it, and I was at best -0 on the comprehension scope
change.
To me, iteration variable assignment in the current scope is a
non-problem.  So to me the change was mostly useless churn. Little
benefit, little harm.  And not worth fighting when others saw a
benefit.


Fair enough, and by itself this might not have been enough reason to 
make the change. But see below.


However, having made the change to nested scopes, I think we should
stick with them.  Or repeal them.  (I believe there is another way to
isolate iteration names -- see  below).  To me, (b) amounts to half
repealing the nested scope change, making comprehensions half-fowl,
half-fish chimeras.


That depends on how you see it -- to me (b) just means that there's an 
implicit nonlocal[1] to make the assignment have the (desirable) 
side-effect.


The key thing to consider here is whether that side-effect is in fact 
desirable. For me, the side-effect of the comprehension's loop control 
variable was never desirable -- it was just an implementation detail 
leaking out. (And that's different from leaking a regular for-loop's 
control variable -- since we have 'break' (and 'else') there are some 
legitimate use cases. But comprehensions try to be expressions, and 
here the side effect is at best useless and at worst a nasty surprise.)


> and who are familiar with the difference in implementation
> of comprehensions (though not generator expressions) in Python 2
vs. 3.

That I pretty much am, I think.  In Python 2, comprehensions (the
fish)
were, at least in effect, expanded in-line to a normal for loop.
Generator expressions (the fowls) were different.  They were, and
still
are, expanded into a temporary generator function whose return
value is
dropped back into the original namespace.  Python 3 turned
comprehensions (with 2 news varieties thereof) into fowls also,
temporary functions whose return value is dropped back in the
original
namespace.  The result is that a list comprehension is equivalent to
list(generator_ expression), even though, for efficiency, it is not
implemented that way.  (To me, this unification is more a benefit
than
name hiding.)


Right, and this consistency convinced me that the change was worth it. 
I just really like to be able to say "[... for ...]" is equivalent to 
"list(... for ...)", and similar for set and dict.


"A shorthand to list()/dict()/set()" is actually how I thought of 
comprehensions when I studied them. And I was actually using list() in 
my code for some time before I learned of their existence.



(b) proposes to add extra hidden code in and around the temporary
function to partly undo the isolation.


But it just adds a nonlocal declaration. There's always some hidden 
code ('def' and 'return' at the very least).


list comprehensions would no
longer be equivalent to list(generator_expression), unless
generator_expressions got the same treatment, in which case they
would
no longer be equivalent to calling the obvious generator function.
Breaking either equivalence might break someone's code.


Ah, there's the rub! I should probably apologize for not clarifying my 
terminology more. In the context of PEP 572, when I say 
"comprehensions" I include generators! PEP 572 states this explicitly 
(https://github.com/python/peps/blame/master/pep-0572.rst#L201-L202).


Certainly PEP 572 intends to add that implicit nonlocal to both 
comprehensions and generator expressions. (I just got really tired of 
writing that phrase over and over, and at some point I forgot that 
this is only a parenthetical remark added in the PEP's latest 
revision, and not conventional terminology -- alas. :-)


Part (b) of PEP 572 does several things of things to *retain* consistency:

- The target of := lives in the same scope regardless of whether it 
occurs in a comprehension, a generator expression, or just in some 
other expression.


- When it occurs in a comprehension or generator expression, the scope 
is the same regardless of whether it occurs in the "outermost 
iterable" or not.


If we didn't have (b) the target would live in the 
comprehension/genexpr scope if it occurred in a comprehension/genexp 
but outside its "outermost iterable", and in the surrounding scope 
otherwise.


---

How loop variables might be isolated without a nested 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Nick Coghlan]

On 26 June 2018 at 02:27, Guido van Rossum  wrote:
> [This is my one reply in this thread today. I am trying to limit the amount
> of time I spend to avoid another overheated escalation.]

Aye, I'm trying to do the same, and deliberately spending some
evenings entirely offline is helping with that :)

> On Mon, Jun 25, 2018 at 4:44 AM Nick Coghlan  wrote:
>>
>> Right, the proposed blunt solution to "Should I use 'NAME = EXPR' or
>> 'NAME := EXPR'?" bothers me a bit, but it's the implementation
>> implications of parent local scoping that I fear will create a
>> semantic tar pit we can't get out of later.
>
> Others have remarked this too, but it really bother me that you are focusing
> so much on the implementation of parent local scoping rather than on the
> "intuitive" behavior which is super easy to explain -- especially to someone
> who isn't all that familiar (or interested) with the implicit scope created
> for the loop control variable(s). According to Steven (who noticed that this
> is barely mentioned in most tutorials about comprehensions) that is most
> people, however very few of them read python-dev.
>
> It's not that much work for the compiler, since it just needs to do a little
> bit of (new) static analysis and then it can generate the bytecode to
> manipulate closure(s). The runtime proper doesn't need any new
> implementation effort. The fact that sometimes a closure must be introduced
> where no explicit initialization exists is irrelevant to the runtime -- this
> only affects the static analysis, at runtime it's no different than if the
> explicit initialization was inside `if 0`.

One of the things I prize about Python's current code generator is how
many of the constructs can be formulated as simple content-and-context
independent boilerplate removal, which is why parent local scoping (as
currently defined in PEP 572) bothers me: rather than being a new
primitive in its own right, the PEP instead makes the notion of "an
assignment expression in a comprehension or generator expression" a
construct that can't readily decomposed into lower level building
blocks the way that both assignment expressions on their own and
comprehensions and generator expressions on their own can be. Instead,
completely new language semantics arise from the interaction between
two otherwise independent features.

Even changes as complicated as PEP 343's with statement, PEP 380's
yield from, and PEP 492's native coroutines all include examples of
how they could be written *without* the benefit of the new syntax.

By contrast, PEP 572's parent local scoping can't currently be defined
that way. Instead, to explain how the code generator is going to be
expected to handle comprehensions, you have to take the current
comprehension semantics and add two new loops to link up the bound
names correctly::

[item := x for x in items]

becomes:

# Each bound name gets declared as local in the parent scope
if 0:
for item in (): pass
def _list_comp(_outermost_iter):
# Each bound name gets declared as:
#   - nonlocal if outer scope is a function scope
#   - global item if outer scope is a module scope
#   - an error, otherwise
_result = []
for x in _outermost_iter:
_result.append(x)
return _result

_expr_result = _list_comp(items)

This is why my objections would be reduced significantly if the PEP
explicitly admitted that it was defining a new kind of scoping
semantics, and actually made those semantics available as an explicit
"parentlocal NAME" declaration (behind a "from __future__ import
parent_locals" guard), such that the translation of the above example
to an explicitly nested scope could just be the visually
straightforward::

def _list_comp(_outermost_iter):
parentlocal item
_result = []
for x in _outermost_iter:
item = x
_result.append(x)
return _result

_expr_result = _list_comp(items)

That splits up the learning process for anyone trying to really
understand how this particular aspect of Python's code generation
works into two distinct pieces:

- "assignment expressions inside comprehensions and generator
expressions use parent local scoping"
- "parent local scoping works "

If the PEP did that, we could likely even make parent locals work
sensibly for classes by saying that "parent local" for a method
definition in a class body refers to the closure namespace where we
already stash __class__ references for the benefit of zero-arg super
(this would also be a far more robust way of defining private class
variables than name mangling is able to offer).

Having parent locals available as a language level concept (rather
than solely as an interaction between assignment expressions and
implicitly nested scopes) also gets us to a point where
context-independent code thunks that work both at module level and
inside another function can be built as nested 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Eric V. Smith]

On 6/27/2018 7:08 AM, Chris Angelico wrote:

It gets funnier with nested loops. Or scarier. I've lost the ability
to distinguish those two.

def test():
 spam = 1
 ham = 2
 vars = [key1+key2 for key1 in locals() for key2 in locals()]
 return vars

Wanna guess what that's gonna return?


I'm not singling out Chris here, but these discussions would be easier 
to follow and more illuminating if the answers to such puzzles were 
presented when they're posed.


Eric

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Angelico]

On Wed, Jun 27, 2018 at 7:19 PM, Steven D'Aprano  wrote:
> On Wed, Jun 27, 2018 at 05:52:16PM +1000, Chris Angelico wrote:
>
>> def test():
>> a = 1
>> b = 2
>> vars = {key: locals()[key] for key in locals()}
>> return vars
>>
>> What would your intuition say? Should this be equivalent to dict(locals()) ?
>
> That example is so elegant it makes me want to cry.
>
> And not just because you shadowed the vars() builtin *wink*

It gets funnier with nested loops. Or scarier. I've lost the ability
to distinguish those two.

def test():
spam = 1
ham = 2
vars = [key1+key2 for key1 in locals() for key2 in locals()]
return vars

Wanna guess what that's gonna return?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev via Python-Dev]

On 26.06.2018 1:34, Greg Ewing wrote:

Ivan Pozdeev via Python-Dev wrote:
"as" was suggested even before is became a keyword in `with'. ( if 
(re.match(regex,line) as m) is not None:  )


That's not equivalent where/given, though, since it still
has the asymmetry problem.

What do you mean by "asymmetry"? The fact that the first time around, 
it's the expression and after that, the variable?


If that, it's not a "problem". The whole idea is to assign the result of 
a subexpression to something.
If you force any assignments to be outside, it won't be a subexpression 
anymore, but effectively a separate statement -- if not syntactically, 
then visually at least -- both of which are the things the feature's 
purpose is to avoid.


If you seek to force assignments outside, you should've rather suggested 
inline code blocks e.g. like anonymous methods in C# ( { a=foo(); 
b=bar(); return a+b;} ).


Using this assigned result elsewhere in the same expression (akin to 
regex backreferences) is not a part of the basic idea actually.
It depends on the evaluation order (and whether something is evaluated 
at all), so I doubt it should even be allowed -- but even if it is, it's 
a side benefit at best.


--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 05:52:16PM +1000, Chris Angelico wrote:

> def test():
> a = 1
> b = 2
> vars = {key: locals()[key] for key in locals()}
> return vars
> 
> What would your intuition say? Should this be equivalent to dict(locals()) ?

That example is so elegant it makes me want to cry.

And not just because you shadowed the vars() builtin *wink*



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Wed, Jun 27, 2018 at 08:30:00AM +0100, Paul Moore wrote:
> On 27 June 2018 at 07:54, Steven D'Aprano  wrote:
> > Comprehensions already run partly in the surrounding scope.
[...]
> > Given the code shown:
> >
> > def test():
> > a = 1
> > b = 2
> > result = [value for key, value in locals().items()]
> > return result
[...]

> But test() returns [1, 2]. So does that say (as you claim above) that
> "the comprehension ran in the enclosing scope"? Doesn't it just say
> that the outermost iterable runs in the enclosing scope?

I think I was careful enough to only say that this was the same result 
you would get *if* the comprehension ran in the outer scope. Not to 
specifically say it *did* run in the outer scope. (If I slipped up 
anywhere, sorry.)

I did say that the comprehension runs *partly* in the surrounding scope, 
and the example shows that the local namespace in the "... in iterable" 
part is not the same as the (sub)local namespace in the "expr for x in 
..." part.

*Parts* of the comprehension run in the surrounding scope, and parts of 
it run in an implicit sublocal scope inside a hidden function, giving us 
a quite complicated semantics for "comprehension scope":

  [expression for a in first_sequence for b in second ... ]
  |--sublocal-|local-|--sublocal--|

Try fitting *that* in the LEGB (+class) acronym :-)


This becomes quite relevant once we include assignment expressions. To 
make the point that this is not specific to := but applies equally to 
Nick's "given" syntax as well, I'm going to use his syntax:

result = [a for a in (x given x = expensive_function(), x+1, 2*x, x**3)]

Here, the assignment to x runs in the local part. I can simulate that 
right now, using locals, but only outside of a function due to CPython's 
namespace optimization inside functions. (For simplicity, I'm just going 
to replace the call to "expensive_function" with just a constant.)


py> del x
py> [a for a in (locals().__setitem__('x', 2) or x, x+1, 2*x, x**3)]
[2, 3, 4, 8]
py> x
2


This confirms that the first sequence part of the comprehension runs in 
the surrounding local scope.

So far so good. What if we move that assignment one level deep? 
Unfortunately, I can no longer use locals for this simulation, due to a 
peculiarity of the CPython function implementation. But replacing the 
call to locals() with globals() does the trick:

del x
# simulate [b*a for b in (1,) for a in (x given x = 2, x+1, 2*x, x**3)]
[b*a for b in (1,) for a in (globals().__setitem__('x', 2) or x, x+1, 2*x, 
x**3)]


That also works. But the problem comes if the user tries to assign to x 
in both the local and a sublocal section:

# no simulation here, sorry
[b*a for b in (x given x = 2, x**2) for a in (x given x = x + 1, x**3)]

That looks like it should work. You're assigning to the same x in two 
parts of the same expression. Where's the problem?

But given the "implicit function" implementation of comprehensions, I 
expect that this ought to raise an UnboundLocalError. The local scope 
part is okay:

# needs a fixed-width font for best results
[b*a for b in (x given x = 2, x**2) for a in (x given x = x + 1, x**3)]
..|-local part|.|sublocal part|

but the sublocal part defines x as a sublocal variable, shadowing the 
surrounding local x, then tries to get a value for that sublocal x 
before it is defined.

If we had assignment expressions before generator expressions and 
comprehensions, I don't think this would have been the behaviour we 
desired.

(We might, I guess, accept it as an acceptable cost of the implicit 
function implementation. But we surely wouldn't argue for this 
complicated scoping behaviour as a good thing in and of itself.)

In any case, we can work around this (at some cost of clarity and 
unobviousness) by changing the name of the variable. Not a big burden 
when the variable is a single character x:

[b*a for b in (x given x = 2, x**2) for a in (y given y = x + 1, y**3)]

but if x is a more descriptive name, that becomes more annoying. 
Nevermind, it is a way around this.

Or we could Just Make It Work by treating the entire comprehension as 
the same scope for assignment expressions. (I stress, not for the loop 
variable.) Instead of having to remember which bits of the comprehension 
run in which scope, we have a conceptually much simpler rule:

- comprehensions are expressions, and assignments inside them
  bind to the enclosing local scope, just like other expressions:

- except for the loop variables, which are intentionally
  encapsulated inside the comprehension and don't "leak".

The *implementation details* of how that works are not conceptually 
relevant. We may or may not want to advertise the fact that 
comprehensions use an implicit hidden function to do the encapsulation, 
and implicit hidden nonlocal to undo the effects of that hidden 
function. Or whatever implementation we happen to use.


> So everybody 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Paul Moore]

On 27 June 2018 at 08:52, Chris Angelico  wrote:
> On Wed, Jun 27, 2018 at 5:30 PM, Paul Moore  wrote:
>> But test() returns [1, 2]. So does that say (as you claim above) that
>> "the comprehension ran in the enclosing scope"? Doesn't it just say
>> that the outermost iterable runs in the enclosing scope?
>
> Yes - because the *outermost iterable* runs in the enclosing scope.
> But suppose you worded it like this:
>
> def test():
> a = 1
> b = 2
> vars = {key: locals()[key] for key in locals()}
> return vars
>
> What would your intuition say? Should this be equivalent to dict(locals()) ?

As I said on python-list, my intuition doesn't apply to locals() - I
simply have no idea what I'd "expect" from that code, other than a
request to go back and write it more clearly :-)

*After* staring at it for a while and trying to interpret it base on
the detailed knowledge I've gained from this thread, I'd say it does
nothing remotely useful, and if you want dict(locals()) you should
write it. (No, test() is not equivalent, because the two instances of
locals() refer to different scopes, but I can't imagine why I'd ever
need to know that outside of solving artificial puzzles like this).

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Chris Angelico]

On Wed, Jun 27, 2018 at 5:30 PM, Paul Moore  wrote:
> But test() returns [1, 2]. So does that say (as you claim above) that
> "the comprehension ran in the enclosing scope"? Doesn't it just say
> that the outermost iterable runs in the enclosing scope?

Yes - because the *outermost iterable* runs in the enclosing scope.
But suppose you worded it like this:

def test():
a = 1
b = 2
vars = {key: locals()[key] for key in locals()}
return vars

What would your intuition say? Should this be equivalent to dict(locals()) ?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Tue, Jun 26, 2018 at 10:54:12AM +1200, Greg Ewing wrote:

> A decision still needs to be made about whether we *want*
> semantics that leak some things but not others.

My sense (or bias, if you prefer) is that the answer to that depends 
on how you word the question. If you talk about "leaking", or 
give examples with trivial 1-character names that look all too 
easy to accidentally clobber, people will say "No":

# Given this:
x = 999
[(x := i)*x for i in (1, 2)]

# should print(x) afterwards result in 4?
   

but if you show a useful example that doesn't look like an accident 
waiting to happen, but a deliberate feature:

# Given this:
previous = 0
[previous + (previous := i) for i in (1, 2, 3)]

# what value would you expect previous to have
# at the completion of the loop?

they'll be more receptive to the idea. (If they're not opposed to 
assignment expressions at all.)

Avoiding leading questions is *hard*, and I believe that in general 
people don't know what they want until they've got it. I say that from 
considering all the times I've made a radical about face, features which 
I was *sure* would be awful actually turned out to be not awful at all 
-- augmented assignment, for instance.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Paul Moore]

On 27 June 2018 at 07:54, Steven D'Aprano  wrote:
> Comprehensions already run partly in the surrounding scope.
>
> I tried to take a survey of people on the Python-List mailing list, so
> see what their expectations of comprehension scope was. Disappointingly,
> not many people responded, but those who did, invariably think in terms
> of comprehensions running inside their enclosing scope, like any other
> expression:
>
> https://mail.python.org/pipermail/python-list/2018-June/734838.html
>
> (Please excuse the doubled-up posts, some misconfigured news server is
> periodically sending duplicate posts.)
>
> (Oh and ignore my comment about Python 2 -- I was thinking of
> something else.)
>
> Given the code shown:
>
> def test():
> a = 1
> b = 2
> result = [value for key, value in locals().items()]
> return result
>
>
> nobody suggested that the result ought to be the empty list, which is
> what you should get if the comprehension ran in its own scope. Instead,
> they all expected some variation of [1, 2], which is what you would get
> if the comprehension ran in the enclosing scope.
>
> A decade or more since generator expressions started running in their
> own half-local-half-sublocal scope, people still think of scoping in
> terms of LEGB and don't think of comprehensions as running in their own
> scope *except* to the very limited degree that sometimes they are either
> surprised or pleased that "the loop variable doesn't leak".

But test() returns [1, 2]. So does that say (as you claim above) that
"the comprehension ran in the enclosing scope"? Doesn't it just say
that the outermost iterable runs in the enclosing scope?

So everybody expected the actual behaviour? (Disclaimer: in my
response, I said that I had no clear expectation, which I stand by -
locals() exposes implementation details that I don't normally feel
that I need to know - but certainly the majority of respondents
expected 1 and 2 to appear).

On the other hand,

>>> def test2():
... a = 1
... b = 2
... result = [locals().items() for v in 'a']
... return result
...
>>> test2()
[dict_items([('v', 'a'), ('.0', )])]

and I bet no-one would have expected that if you'd posed that question
(I certainly wouldn't). Although some might have said [('v', 'a')]. I
suspect some would have expected a and b to appear there too, but
that's just a guess...

So yes, it's likely that people would have found the current behaviour
unexpected in respect of locals(). But I imagine most people only care
about the effective results when referencing variables, and

>>> def test3():
... a = 1
... b = 2
... result = [a for v in (1,)]
... return result
...
>>> test3()
[1]

i.e., thanks to scope nesting, you can still reference locals from the
enclosing scope.

The problem is that := allows you to *change* values in a scope, and
at that point you need to know *which* scope. So to that extent, the
locals() question is important. However, I still suspect that most
people would answer that they would like := to assign values *as if*
they were in the enclosing scope, which is not really something that I
think people would express in answer to a question about locals().
This can be achieved with an implicit "nonlocal" (and some extra
shenanigans if the enclosing scope has a nonlocal or global
declaration itself). Which, AIUI, is what the current proposal tries
to do.

IMO, the big question over the current PEP 572 proposal is whether it
goes too far in the direction of "do what I mean". Superficially, the
semantics are pretty clearly "what people would expect", and indeed
that's been the whole focus recently to capture and satisfy *expected*
behaviour. But there are edge cases (there always are when you work
from messy real-world requirements rather than nice clean mathematical
definitions ;-)) and the question is essentially whether any of those
are bad enough to be an issue.

I'm starting to feel that they aren't, and I'm moving towards a
cautious +0 (or even +1) on the proposal.
Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Steven D'Aprano]

On Tue, Jun 26, 2018 at 05:42:43AM +1000, Chris Angelico wrote:

> So. sublocal scopes, like in the earliest versions of PEP 572?
> 
> The wheel turns round and round, and the same spokes come up.

It isn't as if comprehensions (and generator expressions) run in a 
proper separate scope. It is more half-and-half, sometimes it is 
seperate, sometimes it isn't:


py> def show_subscope():
... a, b = 1, 2
... print("Comprehension scope, Part A")
... print(next(locals() for x in (1,)))
... print("Comprehension scope, Part B")
... print(next(obj for obj in (locals(),)))
...
py> show_subscope()
Comprehension scope, Part A
{'x': 1, '.0': }
Comprehension scope, Part B
{'b': 2, 'a': 1}


Comprehensions already run partly in the surrounding scope.

I tried to take a survey of people on the Python-List mailing list, so 
see what their expectations of comprehension scope was. Disappointingly, 
not many people responded, but those who did, invariably think in terms 
of comprehensions running inside their enclosing scope, like any other 
expression:

https://mail.python.org/pipermail/python-list/2018-June/734838.html

(Please excuse the doubled-up posts, some misconfigured news server is 
periodically sending duplicate posts.)

(Oh and ignore my comment about Python 2 -- I was thinking of 
something else.)

Given the code shown:

def test():
a = 1
b = 2
result = [value for key, value in locals().items()]
return result


nobody suggested that the result ought to be the empty list, which is 
what you should get if the comprehension ran in its own scope. Instead, 
they all expected some variation of [1, 2], which is what you would get 
if the comprehension ran in the enclosing scope.

A decade or more since generator expressions started running in their 
own half-local-half-sublocal scope, people still think of scoping in 
terms of LEGB and don't think of comprehensions as running in their own 
scope *except* to the very limited degree that sometimes they are either 
surprised or pleased that "the loop variable doesn't leak".

For example:

http://nbviewer.jupyter.org/github/rasbt/python_reference/blob/master/tutorials/scope_resolution_legb_rule.ipynb

doesn't mention comprehensions until the very end, almost in passing, 
and doesn't describe them as a separate scope at all. Rather, they are 
described as using closures "to prevent the for-loop variable to cut 
[sic] into the global namespace."

This doesn't mention comprehension subscope at all:

https://www.python-course.eu/python3_global_vs_local_variables.php

Even the official documentation doesn't explicitly state that 
comprehensions are a separate scope:

https://docs.python.org/3/reference/executionmodel.html#resolution-of-names

rather leaving it to an after thought, to mention in passing almost as 
if it were an implementation-dependent accident, that comprehensions 
cannot see variables defined in any surrounding class scope.

Aside from the loop variable (which PEP 572 will not change!) I see no 
evidence that the average non-core developer Python programmer considers 
comprehensions as a separate scope, or wants them to be a separate 
scope. Regardless of comprehensions being implicitly wrapped in a 
function or not, the average developer doesn't want the loop variable to 
"leak", and that's as far as their consideration has needed to go until 
now. But when pressed to explicitly consider the scope inside a 
comprehension, the evidence I have seen is that they consider it the 
same as the local scope surrounding it.

Which is not wrong, as can be seen from the example above.

Unlike the loop variable, I don't believe that assignment-expression 
bindings quote-unquote "leaking" from comprehensions will come as a 
surprise. On the contrary -- given that Nick et al have gone to great 
lengths to ensure that as a first approximation, comprehensions are 
equivalent to a simple for-loop running in the current scope:

result = [expr for a in seq]

# is almost the same as
result = []
for a in seq:
result.append(expr)

I expect that people will be surprised if explicit, non-loop 
variable assignments *don't* occur in the current scope.

If all that takes to implement is something like an implicit "nonlocal", 
that's hardly worse than the implicit functions already used.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Terry Reedy]

On 6/26/2018 10:36 PM, Guido van Rossum wrote:

[This is my one response today]


Thank you for clearly presenting how you see 'comprehension', 'generator 
expression' and by implication 'equivalent code'.  The latter can either 
be a definition or an explanation.  The difference is subtle but real, 
and, I believe, the essence of the disagreement over iteration 
variables.  If the code equivalent to a comprehension is its definition, 
like a macro expansion, then survival of the iteration variable is to be 
expected.  If the equivalent code is an explanation of the *result* of 
evaluating a *self-contained expression*, then leakage is easily seen a 
wart, just as leakage of temporaries from any other expression would be.


My interpretation of what you say below is that you always wanted, for 
instance, [i*i for i in iterable] == [j*j for j in iterable] to be true, 
and saw the leakage making this not quite true as a wart.  In other 
words, within comprehensions (including generator expressions) 
iterations names should be regarded as dummy placeholders and not part 
of the value.


If this is correct, the list comprehension syntax could have been
  [\0 * \0 for \0 in iterable]
with  \1, \2, ... used as needed. (I am using the regex back-reference 
notation in a way similar to the use of str.format forward reference 
notation.)


I will stop here for now, as it is 1:30 am for me.

Terry

On Mon, Jun 25, 2018 at 12:40 PM Terry Reedy > wrote:


On 6/24/2018 7:25 PM, Guido van Rossum wrote:
 > I'd wager that the people who might be most horrified about it

the (b) scoping rule change

 > would be people who feel strongly that the change to the
 > comprehension scope rules in Python 3 is a big improvement,

I might not be one of those 'most horrified' by (b), but I increasingly
don't like it, and I was at best -0 on the comprehension scope change.
To me, iteration variable assignment in the current scope is a
non-problem.  So to me the change was mostly useless churn.  Little
benefit, little harm.  And not worth fighting when others saw a benefit.


Fair enough, and by itself this might not have been enough reason to 
make the change. But see below.


However, having made the change to nested scopes, I think we should
stick with them.  Or repeal them.  (I believe there is another way to
isolate iteration names -- see  below).  To me, (b) amounts to half
repealing the nested scope change, making comprehensions half-fowl,
half-fish chimeras.


That depends on how you see it -- to me (b) just means that there's an 
implicit nonlocal[1] to make the assignment have the (desirable) 
side-effect.


The key thing to consider here is whether that side-effect is in fact 
desirable. For me, the side-effect of the comprehension's loop control 
variable was never desirable -- it was just an implementation detail 
leaking out. (And that's different from leaking a regular for-loop's 
control variable -- since we have 'break' (and 'else') there are some 
legitimate use cases. But comprehensions try to be expressions, and here 
the side effect is at best useless and at worst a nasty surprise.)


 > and who are familiar with the difference in implementation
 > of comprehensions (though not generator expressions) in Python 2
vs. 3.

That I pretty much am, I think.  In Python 2, comprehensions (the fish)
were, at least in effect, expanded in-line to a normal for loop.
Generator expressions (the fowls) were different.  They were, and still
are, expanded into a temporary generator function whose return value is
dropped back into the original namespace.  Python 3 turned
comprehensions (with 2 news varieties thereof) into fowls also,
temporary functions whose return value is dropped back in the original
namespace.  The result is that a list comprehension is equivalent to
list(generator_ expression), even though, for efficiency, it is not
implemented that way.  (To me, this unification is more a benefit than
name hiding.)


Right, and this consistency convinced me that the change was worth it. I 
just really like to be able to say "[... for ...]" is equivalent to 
"list(... for ...)", and similar for set and dict.


(b) proposes to add extra hidden code in and around the temporary
function to partly undo the isolation.


But it just adds a nonlocal declaration. There's always some hidden code 
('def' and 'return' at the very least).


list comprehensions would no
longer be equivalent to list(generator_expression), unless
generator_expressions got the same treatment, in which case they would
no longer be equivalent to calling the obvious generator function.
Breaking either equivalence might break someone's code.


Ah, there's the rub! I should probably apologize for not clarifying my 
terminology more. In the context of PEP 572, when I say "comprehensions" 
I include 

Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Greg Ewing]

Steven D'Aprano wrote:
":=" is the second most common syntax used for assignment in common 
programming languages,


Yes, but it represents an *ordinary* assigment in those
languages. The special way that's being proposed to use
it in Python is not obvious.

--
Greg

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com