Re: [Python-Dev] About [].append == [].append
On 2018-06-23 03:50, Steven D'Aprano wrote: I think it is more important that builtin methods and Python methods behave the same. +1 This inconsistency is the *real* problem here. It's one little extra complication to merging those classes (which was proposed in PEP 575, 576 and 579). ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] About [].append == [].append
23.06.18 10:27, Jeroen Demeyer пише: On 2018-06-23 03:50, Steven D'Aprano wrote: I think it is more important that builtin methods and Python methods behave the same. +1 This inconsistency is the *real* problem here. It's one little extra complication to merging those classes (which was proposed in PEP 575, 576 and 579). +1 too. But I think the right solution should be opposite: reverting issue1350060 changes and making all methods equality be based on the identity of __self__. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] About [].append == [].append
Hi, On 23 June 2018 at 10:54, Serhiy Storchaka wrote: > +1 too. But I think the right solution should be opposite: reverting > issue1350060 changes and making all methods equality be based on the > identity of __self__. The arguments in this thread are the kind of discussion I was looking for when I asked for one in https://bugs.python.org/issue1350060 :-) Better 12 years late than never. Fwiw I also tend to think nowadays that the right solution should be the opposite, like you say. A bientôt, Armin. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 23.06.2018 5:46, Steven D'Aprano wrote:
On Fri, Jun 22, 2018 at 10:59:43AM -0700, Michael Selik wrote:
I've started testing the proposed syntax when I teach. I don't have a
large
sample yet, but most students either dislike it or don't appreciate the
benefits. They state a clear preference for shorter, simpler lines at the
consequence of more lines of code.
Of course they do -- they're less fluent at reading code. They don't
have the experience to judge good code from bad.
The question we should be asking is, do we only add features to Python
if they are easy for beginners? It's not that I especially want to add
features which *aren't* easy for beginners, but Python isn't Scratch and
"easy for beginners" should only be a peripheral concern.
Python's design principles are expressed in the Zen. They rather focus
on being no more complex than absolutely necessary, without prioritizing
either beginners or old-timers ("simple is better than complex",
"complex is better than complicated").
This is partly because students, lacking the experience to instantly
recognize larger constructs, prefer a more concrete approach to
coding. "Good code" is code where the concrete behaviour is more
easily understood. As a programmer gains experience, s/he learns to
grok more complex expressions, and is then better able to make use of
the more expressive constructs such as list comprehensions.
I don't think that's the only dynamic going on here. List comprehensions
are more expressive, but also more declarative and in Python they have nice
parallels with SQL and speech patterns in natural language. The concept of
a comprehension is separate from its particular expression in Python. For
example, Mozilla's array comprehensions in Javascript are/were ugly [0].
Mozilla's array comprehensions are almost identical to Python's, aside
from a couple of trivial differences:
evens = [for (i of numbers) if (i % 2 === 0) i];
compared to:
evens = [i for i in numbers if (i % 2 == 0)]
- the inexplicable (to me) decision to say "for x of array" instead of
"for x in array";
- moving the expression to the end, instead of the beginning.
The second one is (arguably, though not by me) an improvement, since it
preserves a perfect left-to-right execution order within the
comprehension.
Students who are completely new to programming can see the similarity of
list comprehensions to spoken language.
o_O
I've been using comprehensions for something like a decade, and I can't
:-)
The closest analogy to comprehensions I know of is set builder notation
in mathematics, which is hardly a surprise. That's where Haskell got the
inspiration from, and their syntax is essentially an ASCIIfied version
of set builder notation:
Haskell: [(i,j) | i <- [1,2], j <- [1..4]]
Maths: {(i,j) : i ∈ {1, 2}, j ∈ {1...4}}
I teach secondary school children maths, and if there's a plain English
natural language equivalent to list builder notation, neither I nor any
of my students, nor any of the text books I've read, have noticed it.
--
Regards,
Ivan
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We now have C code coverage!
On 22 June 2018 at 23:21, Brett Cannon wrote: > Thanks to a PR from Ammar Askar we now run Python under lcov as part of the > code coverage build. And thanks to codecov.io automatically merging code > coverage reports we get a complete report of our coverage (the first results > of which can now be seen at https://codecov.io/gh/python/cpython). > > And funny enough the coverage average changed less than 1%. :) Nice! One thing I noticed, code that's Windows-specific isn't covered. I assume that's because the coverage reports are based on runs of the test suite on Linux. Is it possible to merge in data from the Windows test runs? If not, what's the best way to address this? Should we be mocking things to attempt to test Windows-specific code even on Linux, or should we simply accept that we're not going to achieve 100% coverage and not worry about it? Paul ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] About [].append == [].append
On 6/23/2018 4:54 AM, Serhiy Storchaka wrote: 23.06.18 10:27, Jeroen Demeyer пише: On 2018-06-23 03:50, Steven D'Aprano wrote: I think it is more important that builtin methods and Python methods behave the same. +1 This inconsistency is the *real* problem here. It's one little extra complication to merging those classes (which was proposed in PEP 575, 576 and 579). +1 too. But I think the right solution should be opposite: reverting issue1350060 changes and making all methods equality be based on the identity of __self__. We 3, and Armin Rigo, it appears, agree that equivalent C and Python coded functions should act the same in comparisons. Abstractly, in English, one might say that a.m equals b.m is true if they perform the same action and have the same effect. The problem, even after adding a stipulation of same type, is that the nature of 'action and effect' are different for pure function methods versus mutation methods. Since pure functions ignore the identify of a and b, so should a.pure_func == b.pure_func. a == b is the right test. For example, a.bit_length == b.bit_length should be true, not false, if a == b. On the other hand, identify is essential for mutation methods, so 'a is b' is the right test, as for .append. But without functions having an 'I mutate' flag, '==' cannot know which comparison of a and b to use, so we have to choose one, and the choice should not depend on the coding language. 'a == b' leads to false positives; 'a is b' leads to false negatives. I don't know how method comparison is actually used, if ever, but false negatives seem safer to me. So I currently agree with Serhiy's choice. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We now have C code coverage!
On 6/22/2018 9:40 PM, Nathaniel Smith wrote: On Fri, Jun 22, 2018 at 6:16 PM, Terry Reedy wrote: I discovered the answer to 2. by shift-clicking on a text_x file to see their coverage report for the file. The colors actually do reflect the test lines executed. codecov.io excludes gui tests*, so the reported coverage for tkinter, idlelib, and turtle is deceptive and bogus, and under-reports the total cpython coverage by a percent or two. It would be better to exclude these modules. * I assume that codecov.io uses linux servers. I have read that there are programs that simulate X-Windows so that gui code will execute without actual terminals. Codecov.io doesn't run any tests itself; it's just a service for aggregation and reporting. The coverage information is being gathered while running CPython's regular CI tests, and then uploaded to codecov.io to view. Thank you for the information. So if you want to run the gui tests -- which seems like a good idea if possible! -- then the way to do that would be to make them run as part of the regular Travis/Appveyor/VSTS checks. I have suggested that, and before that, the same for buildbots. The reality is that tkinter, IDLE, or turtle could be disabled on *nix by regressions and the official testing would not notice. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We now have C code coverage!
On Sat, Jun 23, 2018 at 11:31 AM, Terry Reedy wrote: > I have suggested that, and before that, the same for buildbots. The reality > is that tkinter, IDLE, or turtle could be disabled on *nix by regressions > and the official testing would not notice. I'm looking into enabling the GUI tests on some of the CI hosts today, not sure how far I'll make it with that. GUI tests are currently enabled on my Gentoo [1] and Windows [2] builders, and have been for a couple of years at this point; I'm not sure if any other builders are running GUI tests. [1] http://buildbot.python.org/all/#/workers/6 [2] http://buildbot.python.org/all/#/workers/11 -- Zach ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We now have C code coverage!
On 23.06.2018 13:52, Paul Moore wrote: On 22 June 2018 at 23:21, Brett Cannon wrote: Thanks to a PR from Ammar Askar we now run Python under lcov as part of the code coverage build. And thanks to codecov.io automatically merging code coverage reports we get a complete report of our coverage (the first results of which can now be seen at https://codecov.io/gh/python/cpython). And funny enough the coverage average changed less than 1%. :) Nice! One thing I noticed, code that's Windows-specific isn't covered. I assume that's because the coverage reports are based on runs of the test suite on Linux. Is it possible to merge in data from the Windows test runs? If not, what's the best way to address this? Should we be mocking things to attempt to test Windows-specific code even on Linux, or should we simply accept that we're not going to achieve 100% coverage and not worry about it? AFAICS lcov is based on gcov which is GCC-specific. Paul ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru -- Regards, Ivan ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We now have C code coverage!
On 6/23/2018 1:09 PM, Zachary Ware wrote: On Sat, Jun 23, 2018 at 11:31 AM, Terry Reedy wrote: I have suggested that, and before that, the same for buildbots. The reality is that tkinter, IDLE, or turtle could be disabled on *nix by regressions and the official testing would not notice. I'm looking into enabling the GUI tests on some of the CI hosts today, not sure how far I'll make it with that. GUI tests are currently enabled on my Gentoo [1] and Windows [2] builders, and have been for a couple of years at this point; I'm not sure if any other builders are running GUI tests. I noticed your Gentoo builders running gui tests some years ago. When I re-checked perhaps a year ago, they either were not running or seem to have skipped test_idle, maybe the former. [1] http://buildbot.python.org/all/#/workers/6 [2] http://buildbot.python.org/all/#/workers/11 Rechecking now, on Gentoo test_idle appears and passed on these 3.6 and 3.7 pages http://buildbot.python.org/all/#/builders/82/builds/414/steps/5/logs/stdio Neither Firefox nor Edge can find 'test_idle' on these 3.x pages http://buildbot.python.org/all/#/builders/103/builds/1149/steps/5/logs/stdio http://buildbot.python.org/all/#/builders/99/builds/1130/steps/4/logs/stdio test_tk appears on 1130 but not 1149 For your 8.1 bot: test_idle passed for 3.7 http://buildbot.python.org/all/#/builders/133/builds/339/steps/3/logs/stdio but does is not found on this 3.x page (neither is 'test_tk') http://buildbot.python.org/all/#/builders/12/builds/991/steps/3/logs/stdio Both Appveyor and my machine run test_idle when running the full 3.x test suite. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] About [].append == [].append
Let's make it so. On Sat, Jun 23, 2018, 08:53 Terry Reedy wrote: > On 6/23/2018 4:54 AM, Serhiy Storchaka wrote: > > 23.06.18 10:27, Jeroen Demeyer пише: > >> On 2018-06-23 03:50, Steven D'Aprano wrote: > >>> I think it is more important that builtin methods and Python methods > >>> behave the same. > >> > >> +1 > >> > >> This inconsistency is the *real* problem here. It's one little extra > >> complication to merging those classes (which was proposed in PEP 575, > >> 576 and 579). > > > > +1 too. But I think the right solution should be opposite: reverting > > issue1350060 changes and making all methods equality be based on the > > identity of __self__. > > We 3, and Armin Rigo, it appears, agree that equivalent C and Python > coded functions should act the same in comparisons. Abstractly, in > English, one might say that a.m equals b.m is true if they perform the > same action and have the same effect. > > The problem, even after adding a stipulation of same type, is that the > nature of 'action and effect' are different for pure function methods > versus mutation methods. Since pure functions ignore the identify of a > and b, so should a.pure_func == b.pure_func. a == b is the right test. > For example, a.bit_length == b.bit_length should be true, not false, if > a == b. On the other hand, identify is essential for mutation methods, > so 'a is b' is the right test, as for .append. > > But without functions having an 'I mutate' flag, '==' cannot know which > comparison of a and b to use, so we have to choose one, and the choice > should not depend on the coding language. 'a == b' leads to false > positives; 'a is b' leads to false negatives. I don't know how method > comparison is actually used, if ever, but false negatives seem safer to > me. So I currently agree with Serhiy's choice. > > -- > Terry Jan Reedy > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We now have C code coverage!
On Sat, Jun 23, 2018 at 2:20 PM, Terry Reedy wrote:
> Rechecking now, on Gentoo
>
> test_idle appears and passed on these 3.6 and 3.7 pages
> http://buildbot.python.org/all/#/builders/82/builds/414/steps/5/logs/stdio
>
> Neither Firefox nor Edge can find 'test_idle' on these 3.x pages
> http://buildbot.python.org/all/#/builders/103/builds/1149/steps/5/logs/stdio
> http://buildbot.python.org/all/#/builders/99/builds/1130/steps/4/logs/stdio
>
> test_tk appears on 1130 but not 1149
>
> For your 8.1 bot: test_idle passed for 3.7
> http://buildbot.python.org/all/#/builders/133/builds/339/steps/3/logs/stdio
>
> but does is not found on this 3.x page (neither is 'test_tk')
> http://buildbot.python.org/all/#/builders/12/builds/991/steps/3/logs/stdio
Click the magnifying glass icon ("load all data for use with the
browser search tool") at the upper right of the console pane and try
again; each of the above is present and passed. This does
unfortunately seem to be another case of non-intuitive UI from
buildbot.
> Both Appveyor and my machine run test_idle when running the full 3.x test
> suite.
I am pleasantly surprised that AppVeyor does appear to be running the
GUI tests :)
--
Zach
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python3.7 backwards incompatible
Python 3.7 in the status of RC, but I do not see information about the fact that python3.7 is backwards incompatible with python3.5. $ ~ python3.5Python 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609] on linuxType "help", "copyright", "credits" or "license" for more information.>>> from typing import List>>> class MyList(list):... pass... >>> issubclass(MyList, List)True>>> issubclass(List, MyList)False>>> $ ~ python3.7Python 3.7.0rc1+ (heads/3.7:3747dd16d5, Jun 22 2018, 22:53:42) [GCC 5.4.0 20160609] on linuxType "help", "copyright", "credits" or "license" for more information.>>> from typing import List>>> class MyList(list):... pass... >>> issubclass(MyList, List)True>>> issubclass(List, MyList)Traceback (most recent call last): File "", line 1, in TypeError: issubclass() arg 1 must be a class>>> And so with all types of "typing" module.This breaks down many libraries that use annotations as part of the functionality if we now can't use typing.* into issubclass function.___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3.7 backwards incompatible
First, the typing module is still provisional, so there is no strict backwards compatibility guarantee. With that out of the way, I can reproduce your problem, and I assume it's caused by the implementation of PEP 560, which is meant to speed up the typing module (among other goals). I'm wondering about your claim that this breaks many "libraries that use annotations". Most code using annotations would not need to use issubclass() in this way. Where exactly did you encounter this? I'm CC'ing Ivan Levkivskyi, who knows the relevant code better and will be able to explain whether this is an oversight or an intentional regression. On Sat, Jun 23, 2018 at 2:59 PM Rokker Ruslan wrote: > Python 3.7 in the status of RC, but I do not see information about the > fact that python3.7 is backwards incompatible with python3.5. > > $ ~ python3.5 > Python 3.5.2 (default, Nov 23 2017, 16:37:01) > [GCC 5.4.0 20160609] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> from typing import List > >>> class MyList(list): > ... pass > ... > >>> issubclass(MyList, List) > True > >>> issubclass(List, MyList) > False > >>> > > $ ~ python3.7 > Python 3.7.0rc1+ (heads/3.7:3747dd16d5, Jun 22 2018, 22:53:42) > [GCC 5.4.0 20160609] on linux > Type "help", "copyright", "credits" or "license" for more information. > >>> from typing import List > >>> class MyList(list): > ... pass > ... > >>> issubclass(MyList, List) > True > >>> issubclass(List, MyList) > Traceback (most recent call last): > File "", line 1, in > TypeError: issubclass() arg 1 must be a class > >>> > > And so with all types of "typing" module. > This breaks down many libraries that use annotations as part of the > functionality if we now can't use typing.* into issubclass function. > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We now have C code coverage!
On 6/23/2018 5:48 PM, Zachary Ware wrote:
On Sat, Jun 23, 2018 at 2:20 PM, Terry Reedy wrote:
Rechecking now, on Gentoo
test_idle appears and passed on these 3.6 and 3.7 pages
http://buildbot.python.org/all/#/builders/82/builds/414/steps/5/logs/stdio
Neither Firefox nor Edge can find 'test_idle' on these 3.x pages
http://buildbot.python.org/all/#/builders/103/builds/1149/steps/5/logs/stdio
http://buildbot.python.org/all/#/builders/99/builds/1130/steps/4/logs/stdio
test_tk appears on 1130 but not 1149
For your 8.1 bot: test_idle passed for 3.7
http://buildbot.python.org/all/#/builders/133/builds/339/steps/3/logs/stdio
but does is not found on this 3.x page (neither is 'test_tk')
http://buildbot.python.org/all/#/builders/12/builds/991/steps/3/logs/stdio
Click the magnifying glass icon ("load all data for use with the
browser search tool") at the upper right of the console pane and try
again; each of the above is present and passed. This does
unfortunately seem to be another case of non-intuitive UI from
buildbot.
Presenting data on the screen that cannot be found is terrible. With
Edge, I had to erase and retype the search word also. With Firefox,
that sometimes worked without pressing the magnifier. I thought my copy
of Firefox was broken until I tried Edge also.
Both Appveyor and my machine run test_idle when running the full 3.x test
suite.
I am pleasantly surprised that AppVeyor does appear to be running the
GUI tests :)
--
Terry Jan Reedy
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3.7 backwards incompatible
This particular breakage is explicitly listed in PEP 560, see an example with List and List[int] in https://www.python.org/dev/peps/pep-0560/#backwards-compatibility-and-impact-on-users-who-don-t-use-typing In general, isinstance() with typing types should be avoided when possible (Mark Shannon who is the BDFL delegate for PEP 484 wanted to prohibit it completely, but in the end we kept only the bare minimum, like your first example). When designing/implementing PEP 560 I realised it will be impossible to keep 100% backwards compatibility. I tried to preserve 99% of public APIs, but since isinstance() is already discouraged, it fell into remaining 1%. A possible workaround is to use `typing_inspect` library on PyPI (disclaimer: I originally wrote it). You can use `get_origin()` function to extract the runtime class that corresponds to a given type. It works with both 3.6 and 3.7 and tries its best to return the relevant runtime class (with few exceptions, see docstring), for every version, for example on Python 3.7 `get_origin(List[int])` return `list`. The return of get_origin() should be usable in all runtime context including `isinstance()`. Btw, I actually like the new behaviour. After PEP 560 types are no more classes, which emphasises that they should be used in static context, if one wants to do something in runtime, one needs to use an explicit conversion to a runtime class. -- Ivan On 23 June 2018 at 23:10, Guido van Rossum wrote: > First, the typing module is still provisional, so there is no strict > backwards compatibility guarantee. > > With that out of the way, I can reproduce your problem, and I assume it's > caused by the implementation of PEP 560, which is meant to speed up the > typing module (among other goals). > > I'm wondering about your claim that this breaks many "libraries that use > annotations". Most code using annotations would not need to use > issubclass() in this way. Where exactly did you encounter this? > > I'm CC'ing Ivan Levkivskyi, who knows the relevant code better and will be > able to explain whether this is an oversight or an intentional regression. > > On Sat, Jun 23, 2018 at 2:59 PM Rokker Ruslan > wrote: > >> Python 3.7 in the status of RC, but I do not see information about the >> fact that python3.7 is backwards incompatible with python3.5. >> >> $ ~ python3.5 >> Python 3.5.2 (default, Nov 23 2017, 16:37:01) >> [GCC 5.4.0 20160609] on linux >> Type "help", "copyright", "credits" or "license" for more information. >> >>> from typing import List >> >>> class MyList(list): >> ... pass >> ... >> >>> issubclass(MyList, List) >> True >> >>> issubclass(List, MyList) >> False >> >>> >> >> $ ~ python3.7 >> Python 3.7.0rc1+ (heads/3.7:3747dd16d5, Jun 22 2018, 22:53:42) >> [GCC 5.4.0 20160609] on linux >> Type "help", "copyright", "credits" or "license" for more information. >> >>> from typing import List >> >>> class MyList(list): >> ... pass >> ... >> >>> issubclass(MyList, List) >> True >> >>> issubclass(List, MyList) >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: issubclass() arg 1 must be a class >> >>> >> >> And so with all types of "typing" module. >> This breaks down many libraries that use annotations as part of the >> functionality if we now can't use typing.* into issubclass function. >> ___ >> Python-Dev mailing list >> [email protected] >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/ >> guido%40python.org >> > > > -- > --Guido van Rossum (python.org/~guido) > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 23 June 2018 at 09:06, Greg Ewing wrote:
> Nick Coghlan wrote:
>>
>> x:= f():" implies "x" is already defined as a target somewhere else in
>> the current scope, while "if x := f() given x:" potentially introduces
>> "x" as a new local target
>
>
> N. this is just taking a bad idea and making it
> worse, IMO.
>
> I'm -1 on any contortions designed to allow comprehensions
> to assign to things in outer scopes. All the proposed use
> cases I've seen for this have not improved readability
> over writing a function that does things the usual way.
>
> Can we please leave comprehensions as declarative
> constructs? The best tools do just one thing and do
> it well. These proposals seem to be trying to turn
> comprehensions into swiss army knives.
If PEP 572 was proposing the use of regular local scoping for
assignment expressions in comprehensions, such that they could still
be used to avoiding repeating subexpressions within an iteration, but
couldn't be used to export progress data, or to maintain a partial sum
without having to rely on `locals().get("total", 0)` to provide an
initial value, then I wouldn't be feeling obliged to present an
alternative that offers the same state export capabilities in a more
explicit form.
Given that PEP 572 *is* proposing implicit comprehension state export,
though, then I think it's important to make the case that seeing the
proposed semantics as intuitive is only going to be the case for folks
that have used Python 2 style comprehensions extensively - anyone
that's never encountered the old state-leaking behaviour for iteration
variables is going to be surprised when assignment expressions ignore
the existence of the comprehension scope (even though the iteration
variable pays attention to it).
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 2018-06-22 19:46, Steven D'Aprano wrote: - the inexplicable (to me) decision to say "for x of array" instead of "for x in array"; Believe JavaScript has for…in, but as usual in the language it is broken and they needed a few more tries to get it right. for…of is the latest version and works as expected. -Mike ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 23 June 2018 at 13:48, Steven D'Aprano wrote: > On Sat, Jun 23, 2018 at 12:22:33AM +1000, Nick Coghlan wrote: > [...] >> plenty of >> functional-language-inspired documentation to instead encourage folks >> to view comprehensions as tightly encapsulated declarative container >> construction syntax. > > I can't say I've done a broad survey, but the third-party documentation > I've read on comprehensions typically glosses over the scoping issues > without mentioning them. To the extent that scoping is even hinted at, > comprehensions are treated as expressions which are exactly equivalent > to re-writing them as a for-loop in the current scope. > > This is a typical example, found as the top result on googling for > "python comprehensions": > > https://www.google.com/search?q=python+comprehensions > > http://www.pythonforbeginners.com/basics/list-comprehensions-in-python > > Nothing is mentioned about scope, and it repeats the inaccurate but > simple equivalency: > > for item in list: > if conditional: > expression > > But perhaps that tutorial is too old. Okay this recent one is only a > little more than a year old: > > https://hackernoon.com/list-comprehension-in-python-8895a785550b > > Again, no mention of scoping issues, comprehensions are simply > expressions which presumably run in the same scope as any other > expression. > > I think you over-estimate how many newcomers to Python are even aware > that the scope of comprehensions is something to consider. I put quite a bit of work into making it possible for folks to gloss over the distinction and still come to mostly-correct conclusions about how particular code snippets would behave. I was only able to achieve it because the folks that designed lexical scoping before me had made *read* access to lexical scopes almost entirely transparent, and because generator expressions were designed to fail fast if there was a bug in the expression defining the outermost iterable (which meant that even at class scope, the outermost iterable expression still had access to class level variables, because it was evaluated *outside* the nested scope). *Write* access to lexically nested scopes, by contrast, was omitted entirely from the original lexical scoping design, and when it was later added by https://www.python.org/dev/peps/pep-3104/, it was done using an explicit "nonlocal" declaration statement (expressly akin to "global"), and PEP 3099 explicitly ruled out the use of ":=" to implicitly declare the target name as being non-local. PEP 572 is thus taking the position that: - we now want to make write access to outer scopes implicit (despite PEP 3099 explicitly ruling that out as desired design feature) - but only in comprehensions and generator expressions (not lambda expressions, and not full nested functions) - and only for assignment expressions, not for loop iteration variables - and we want it to implicitly choose between a "global NAME" declaration and a "nonlocal NAME" declaration based on where the comprehension is defined - and this is OK because "nobody" actually understands how comprehensions hide the iteration variable in practice, and "everybody" thinks they're still a simple for loop like they were in Python 2 - the fact that the language reference, the behaviour at class scopes, the disassembly output, and the behaviour in a debugger all indicate that comprehensions are full nested scopes isn't important This level of additional complication and complexity in the scoping semantics simply isn't warranted for such a minor readability enhancement as assignment expressions. Cheers, Nick. P.S. "You did such a good job of minimising the backwards compatibility breakage when we changed the semantics of scoping in comprehensions that we now consider your opinion on reasonable scoping semantics for comprehensions to be irrelevant, because everybody else still thinks they work the same way as they did in Python 2" is such a surreal position for folks to be taking that I'm having trouble working out how to effectively respond to it. Guido has complained that "I keep changing my mind about what I want", but that's not what's actually going on: what I want is to keep folks from taking our already complicated scoping semantics and making it close to impossible for anyone to ever infer how they work from experimentation at the interactive prompt. That goal has pretty much stayed consistent since the parent local scoping proposal was first put forward. What keeps changing is my tactics in pursuing that goal, based on my current perception of what the folks pushing that proposal are actually trying to achieve (which seems to be some combination of "We want to pretend that the Python 3 scoping changes in comprehensions never happened, but we still want to avoid leaking the iteration variables somehow" and "We want to enable new clever tricks with state export from comprehensions and generator expressions"), as well as repeatedly asking myse
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 02:33:59PM +1000, Nick Coghlan wrote: > Given that PEP 572 *is* proposing implicit comprehension state export, "Implicit" and "explicit" are two terms which often get misused to mean "I don't like it" and "I do like it". Making the intentional choice to use an assignment expression is not really "implicit" in any meaningful sense. One might as well complain that "import this" implicitly creates a local variable "this". Well, yes, it does, in a very loose sense, but that's what imports are defined as do and it is the whole purpose for making them. If PEP 572's proposal goes ahead, the behaviour of assignment expressions will be *defined* as creating assignments in the local scope rather than the sublocal comprehension scope. To call that "implicit" is rather like saying that regular assignment is implicit. > though, then I think it's important to make the case that seeing the > proposed semantics as intuitive is only going to be the case for folks > that have used Python 2 style comprehensions extensively - anyone > that's never encountered the old state-leaking behaviour for iteration > variables is going to be surprised when assignment expressions ignore > the existence of the comprehension scope (even though the iteration > variable pays attention to it). You are making the assumption that most people are even aware of "comprehension scope". I don't think that is the case. In my experience, scoping in Python is still typically seen as the LGB rule (locals/globals/builtins). See for example this StackOverflow post from 2016: https://stackoverflow.com/questions/37211910/override-lgb-scope-rule Sometimes people remember the E/N (enclosing function/nonlocal) part. Hardly anyone remembers the C (class) part unless they are actively thinking in terms of code running inside a class definition, and even if they do, they typically aren't sure of exactly how it interacts with the rest. And I predict that even fewer think of comprehensions as a separate scope, except by ommission: they *don't think about* the scope of the loop variable until it bites them. But as Tim Peters has previously discussed, the loop variable is special, and is especially prone to accidental shadowing. That won't be the case for assignment expressions. If there's shadowing going on, it will be deliberate. Aside: I've said before that I'm not a fan of sublocal comprehension scoping, since I personally found it helpful on occasion for the loop variable to be visible outside of the comprehension. But given that the only experience most people apparently had with comprehension scoping was to be bitten by it, I grudgingly accept that encapsulating the loop variable was the right decision to make, even if it personally inconvenienced me more than it saved me. Nor was I the only one: others have been bitten by the change to comprehension scope, see for example: https://www.reddit.com/r/Python/comments/425qmb/strange_python_27_34_difference/ There is no consensus that the change to comprehensions was a good thing or justified. The bottom line is that I don't think people will be surprised by assignment expression scope being local instead of sublocal. Rather I expect that they won't even think about it, until they do, and then *whatever* behaviour we pick, we'll annoy somebody. -- Steve ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On 24 June 2018 at 15:56, Steven D'Aprano wrote: > On Sun, Jun 24, 2018 at 02:33:59PM +1000, Nick Coghlan wrote: > >> Given that PEP 572 *is* proposing implicit comprehension state export, > > "Implicit" and "explicit" are two terms which often get misused to mean > "I don't like it" and "I do like it". > > Making the intentional choice to use an assignment expression is not > really "implicit" in any meaningful sense. No, it's actually implicit: there's an extra "global NAME" or "nonlocal NAME" in the equivalent code for a comprehension that isn't there in the as-written source code, and doesn't get emitted for a regular assignment expression or for the iteration variable in a comprehension - it only shows up due to the defined interaction between comprehensions and assignment expressions. > One might as well complain > that "import this" implicitly creates a local variable "this". Well, > yes, it does, in a very loose sense, but that's what imports are > defined as do and it is the whole purpose for making them. And they behave the same way in every context where they're permitted to appear. > If PEP 572's proposal goes ahead, the behaviour of assignment > expressions will be *defined* as creating assignments in the local scope > rather than the sublocal comprehension scope. To call that "implicit" > is rather like saying that regular assignment is implicit. I do say that regular assignments implicitly declare a name as local. "Python has implicit local variable declarations" is also regularly cited as one of the differences between it and languages that require explicit declarations, like C. Even augmented assignments implicitly declare a name as being a local (hence the infamous UnboundLocalError that arises when you attempt to use an augmented assignment to rebind a name from an outer scope). The problem I have with PEP 572 is that it proposes *breaking that otherwise universal pattern* - instead of having assignment expressions in comprehensions implicitly declare the name as local in the nested comprehension scope, it instead has them: 1. implicitly declare the name as global or as nonlocal in the comprehension (or else raise an exception), depending on the nature of the parent scope where the comprehension is used 2. in the nonlocal reference case, amend the symbol table analysis to act like there was a preceding "if 0:\n for NAME in ():\n pass" in the parent scope (so the compiler knows which outer function scope to target) The rationale being given for why that is OK is: 1. "Everyone" thinks comprehensions are just a for loop (even though that hasn't been true for the better part of a decade, and was never true for generator expressions) 2. If comprehensions are just a for loop, then assignment expressions inside them should be local to the containing scope 3. Therefore the implicit declarations required to tie everything together and allow folks to continue with an incorrect understanding of how comprehensions work aren't really implicit - they're explicit in the inaccurate expansion of the construct! Can you imagine the reaction if anyone other than Guido or Tim was attempting to argue for a change to the language that only makes sense if we grant a completely inaccurate understanding of how a particular language construct works as being a credible starting point? Because that's how this comprehension scoping argument feels to me: Proposal author: "If the language worked in a way other than it does, then this proposal would make perfect sense." Proposal reviewer: "Yes, but it doesn't work that way, it works this way. We deliberately changed it because the old way caused problems." Proposal author: "Ah, but it *used* to work that way, and a lot of people still think it works that way, and we can get the compiler to jump through hoops to pretend it still works that way, except for the parts of the new way that we want to keep." Proposal reviewer: "..." Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)
On Sun, Jun 24, 2018 at 4:33 PM, Nick Coghlan wrote: > On 24 June 2018 at 15:56, Steven D'Aprano wrote: >> On Sun, Jun 24, 2018 at 02:33:59PM +1000, Nick Coghlan wrote: >> >>> Given that PEP 572 *is* proposing implicit comprehension state export, >> >> "Implicit" and "explicit" are two terms which often get misused to mean >> "I don't like it" and "I do like it". >> >> Making the intentional choice to use an assignment expression is not >> really "implicit" in any meaningful sense. > > No, it's actually implicit: there's an extra "global NAME" or > "nonlocal NAME" in the equivalent code for a comprehension that isn't > there in the as-written source code, and doesn't get emitted for a > regular assignment expression or for the iteration variable in a > comprehension - it only shows up due to the defined interaction > between comprehensions and assignment expressions. The implicit "nonlocal NAME" is only because there is an equally implicit function boundary. Why is there a function boundary marked by square brackets? It's not saying "def" or "lambda", which obviously create functions. It's a 'for' loop wrapped inside a list display. What part of that says "hey, I'm a nested function"? So if there's an implicit function, with implicit declaration of a magical parameter called ".0", why can't it have an equally implicit declaration that "spam" is a nonlocal name? ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
