Re: [Python-Dev] Guarantee ordered dict literals in v3.7?
08.11.17 04:33, Nick Coghlan пише: On 8 November 2017 at 11:44, Nick Coghlan wrote: 2. So far, I haven't actually come up with a perturbed iteration implementation that doesn't segfault the interpreter. The dict internals are nicely laid out to be iteration friendly, but they really do assume that you're going to start at index zero, and then iterate through to the end of the array. The bounds checking and pointer validity testing becomes relatively fiddly if you try to push against that and instead start iteration from a point partway through the storage array. In case anyone else wants to experiment with a proof of concept: https://github.com/ncoghlan/cpython/commit/6a8a6fa32f0a9cd71d9078fbb2b5ea44d5c5c14d I think we've probably exhausted the utility of discussing this as a purely hypothetical change, and so the only way to move the discussion forward will be for someone to draft a patch that: 1. Perturbs iteration for regular dicts (it's OK for our purposes if it's still deterministic - it just shouldn't match insertion order the way odict does) 2. Switches keyword args and class body execution namespaces over to odict so the test suite passes again 3. Measures the impact such a change would have on the benchmark suite My experiment is a starting point, but it will still be a fair bit of work to get it from there to a viable proof of concept that can be assessed against the status quo. It may be easy and more efficient to break the order at insertion. 1. Insert in the reversed order. 2. Add at the end or at the begin, changing the order on every insertion. 2. Swap with an arbitrary item. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Enabling depreciation warnings feature code cutoff
On Wed, 8 Nov 2017 11:35:13 +1000 Nick Coghlan wrote: > On 8 November 2017 at 10:03, Guido van Rossum wrote: > > OK, so let's come up with a set of heuristics that does the right thing for > > those cases specifically. I'd say whenever you're executing code from a > > zipfile or some such it's not considered your own code (by default). > > My current preferred heuristic is just to add a new default filter to the > list: > > once::DeprecationWarning:__main__ Special cases are not special enough to break the rules. In other words, I'm -1 on this. Not only does it add complication and inconsistency (bound to catch people by surprise) to an already non-trivial set of default warning settings, but it doesn't even solve any problem that I'm aware of. The idea that __main__ scripts should get special treatment here is entirely gratuitous. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The current dict is not an "OrderedDict"
On Wed, 8 Nov 2017 13:07:12 +1000 Nick Coghlan wrote: > On 8 November 2017 at 07:19, Evpok Padding wrote: > > On 7 November 2017 at 21:47, Chris Barker wrote: > >> if dict order is preserved in cPython , people WILL count on it! > > > > I won't, and if people do and their code break, they'll have only themselves > > to blame. > > Also, what proof do you have of that besides anecdotal evidence ? > > ~27 calendar years of anecdotal evidence across a multitude of CPython > API behaviours (as well as API usage in other projects). > > Other implementation developers don't say "CPython's runtime behaviour > is the real Python specification" for the fun of it - they say it > because "my code works on CPython, but it does the wrong thing on your > interpreter, so I'm going to stick with CPython" is a real barrier to > end user adoption, no matter what the language specification says. Yet, PyPy has no reference counting, and it doesn't seem to be a cause of concern. Broken code is fixed along the way, when people notice. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Clarifying Cygwin support in CPython
Hi folks, As some people here know I've been working off and on for a while to improve CPython's support of Cygwin. I'm motivated in part by a need to have software working on Python 3.x on Cygwin for the foreseeable future, preferably with minimal graft. (As an incidental side-effect Python's test suite--especially of system-level functionality--serves as an interesting test suite for Cygwin itself too.) This is partly what motivated PEP 539 [1], although that PEP had the advantage of benefiting other POSIX-compatible platforms as well (and in fact was fixing an aspect of CPython that made it unfriendly to supporting other platforms). As far as I can tell, the first commit to Python to add any kind of support for Cygwin was made by Guido (committing a contributed patch) back in 1999 [2]. Since then, bits and pieces have been added for Cygwin's benefit over time, with varying degrees of impact in terms of #ifdefs and the like (for the most part Cygwin does not require *much* in the way of special support, but it does have some differences from a "normal" POSIX-compliant platform, such as the possibility for case-insensitive filesystems and executables that end in .exe). I don't know whether it's ever been "officially supported" but someone with a longer memory of the project can comment on that. I'm not sure if it was discussed at all or not in the context of PEP 11. I have personally put in a fair amount of effort already in either fixing issues on Cygwin (many of these issues also impact MinGW), or more often than not fixing issues in the CPython test suite on Cygwin--these are mostly tests that are broken due to invalid assumptions about the platform (for example, that there is always a "root" user with uid=0; this is not the case on Cygwin). In other cases some tests need to be skipped or worked around due to platform-specific bugs, and Cygwin is hardly the only case of this in the test suite. I also have an experimental AppVeyor configuration for running the tests on Cygwin [3], as well as an experimental buildbot (not available on the internet, but working). These currently rely on a custom branch that includes fixes needed for the test suite to run to completion without crashing or hanging (e.g. https://bugs.python.org/issue31885). It would be nice to add this as an official buildbot, but I'm not sure if it makes sense to do that until it's "green", or at least not crashing. I have several other patches to the tests toward this goal, and am currently down to ~22 tests failing. Before I do any more work on this, however, it would be best to once and for all clarify the support for Cygwin in CPython, as it has never been "officially supported" nor unsupported--this way we can avoid having this discussion every time a patch related to Cygwin comes up. I could provide some arguments for why I believe Cygwin should supported, but before this gets too long I'd just like to float the idea of having the discussion in the first place. It's also not exactly clear to me how to meet the standards in PEP 11 for supporting a platform--in particular it's not clear when a buildbot is considered "stable", or how to achieve that without getting necessary fixes merged into the main branch in the first place. Thanks, Erik [1] https://www.python.org/dev/peps/pep-0539/ [2] https://github.com/python/cpython/commit/717d1fdf2acbef5e6b47d9b4dcf48ef1829be685 [3] https://ci.appveyor.com/project/embray/cpython ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] OrderedDict(kwargs) optimization?
On Wednesday, November 8, 2017, Guido van Rossum wrote:
> It seems there must be at least two threads for each topic worth
> discussing at all. Therefore I feel compelled to point to
> https://mail.python.org/pipermail/python-dev/2017-November/150381.html,
> where I state my own conclusion about dict order.
>
>
.
>
> Finally: the dict type should not be endowed with other parts of the
> OrderedDict API, not should other API changes to dict be considered.
>
Is there an opportunity to support a fast cast to OrderedDict from 3.6 dict?
Can it just copy .keys() into the OrderedDict linked list?Or is there more
overhead to the transition?
That'd be great for preserving kwargs' order after a pop() or a del?
def func(**kwargs):
kwargs = OrderedDict(kwargs)
arg2 = kwargs.pop('arg2')
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Clarifying Cygwin support in CPython
On Wed, Nov 8, 2017 at 8:39 AM, Erik Bray wrote: > a platform--in particular it's not clear when a buildbot is considered > "stable", or how to achieve that without getting necessary fixes > merged into the main branch in the first place. I think in this context, "stable" just means "keeps a connection to the buildbot master and doesn't blow up when told to build" :). As such, I'm ready to get you added to the fleet whenever you are. -- Zach ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Add Py_SETREF and Py_XSETREF to the stable C API
Macros Py_SETREF and Py_XSETREF were introduced in 3.6 and backported to all maintained versions ([1] and [2]). Despite their names they are private. I think that they are enough stable now and would be helpful in third-party code. Are there any objections against adding them to the stable C API? [3] [1] https://bugs.python.org/issue20440 [2] https://bugs.python.org/issue26200 [3] https://bugs.python.org/issue31983 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add Py_SETREF and Py_XSETREF to the stable C API
I like these macros! Technically, would it be possible to use an inline function instead of a macro for Python 3.7? Victor 2017-11-08 17:30 GMT+01:00 Serhiy Storchaka : > Macros Py_SETREF and Py_XSETREF were introduced in 3.6 and backported to all > maintained versions ([1] and [2]). Despite their names they are private. I > think that they are enough stable now and would be helpful in third-party > code. Are there any objections against adding them to the stable C API? [3] > > [1] https://bugs.python.org/issue20440 > [2] https://bugs.python.org/issue26200 > [3] https://bugs.python.org/issue31983 > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add Py_SETREF and Py_XSETREF to the stable C API
08.11.17 18:37, Victor Stinner пише: I like these macros! Technically, would it be possible to use an inline function instead of a macro for Python 3.7? No, unless there is a way to pass arguments by reference in C99. Py_SETREF(x, y) is the safe equivalent of x = y; Py_DECREF(x, y); ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: go back to enabling DeprecationWarning by default
Philipp, You seem to have missed Nick's posts where he clearly accepts that a middle ground is necessary. R D Murray is also still unconvinced. (And obviously I myself am against reverting to the behavior from 7 years ago.) If we can't agree on some middle ground, the status quo will be maintained. On Tue, Nov 7, 2017 at 11:47 PM, Philipp A. wrote: > Hi Guido, > > As far as I can see, the general consensus seems to be to turn them back > on in general: The last person to argue against it was Paul Moore, and he > since said: > > “OK, I overstated [that you’re ‘hosed’ by DeprecationWarnings appearing]. > Apologies. My recollection is of a lot more end user complaints when > deprecation warnings were previously switched on than others seem to > remember, but I can't find hard facts, so I'll assume I'm misremembering.” > > Besides, quite some of the problems people mention would only be fixed by > turning them on in general, not with the compromise. > > So I don’t think we need a compromise, right? > > Best, Philipp > > Guido van Rossum schrieb am Mi., 8. Nov. 2017 um > 03:46 Uhr: > >> To cut this thread short, I say we should use Nick's proposal to turn >> these warnings on for __main__ but off elsewhere. (See >> https://mail.python.org/pipermail/python-dev/2017-November/150364.html.) >> >> >> -- >> --Guido van Rossum (python.org/~guido) >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/ >> flying-sheep%40web.de >> > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: go back to enabling DeprecationWarning by default
On Nov 8, 2017, at 08:47, Guido van Rossum wrote: > > You seem to have missed Nick's posts where he clearly accepts that a middle > ground is necessary. R D Murray is also still unconvinced. (And obviously I > myself am against reverting to the behavior from 7 years ago.) If we can't > agree on some middle ground, the status quo will be maintained. I haven’t seen a response to my suggestion, so it’s possible that it got missed in the flurry. With coordination with setuptools, we could: * Re-enable DeprecationWarning by default * Add a simplified API for specifically silencing DeprecationWarnings * Modify setuptools to call this API for generated entry point scripts I think this would mean that most application users would still not see the warnings. The simplified API would be available for handcrafted scripts to call to accomplish the same thing the setuptools enhancement would provide. Developers would see DeprecationWarnings in their development and test environments. The simplified API would be the equivalent of ignore::DeprecationWarning, so with some additional documentation even versions of applications running on versions of Python < 3.7 would still have an “out”. (Yes, the simplified API is just a convenience moving forward.) Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: go back to enabling DeprecationWarning by default
I hadn't seen that, but it requires too much cooperation of library owners. On Wed, Nov 8, 2017 at 10:56 AM, Barry Warsaw wrote: > On Nov 8, 2017, at 08:47, Guido van Rossum wrote: > > > > You seem to have missed Nick's posts where he clearly accepts that a > middle ground is necessary. R D Murray is also still unconvinced. (And > obviously I myself am against reverting to the behavior from 7 years ago.) > If we can't agree on some middle ground, the status quo will be maintained. > > I haven’t seen a response to my suggestion, so it’s possible that it got > missed in the flurry. With coordination with setuptools, we could: > > * Re-enable DeprecationWarning by default > * Add a simplified API for specifically silencing DeprecationWarnings > * Modify setuptools to call this API for generated entry point scripts > > I think this would mean that most application users would still not see > the warnings. The simplified API would be available for handcrafted > scripts to call to accomplish the same thing the setuptools enhancement > would provide. Developers would see DeprecationWarnings in their > development and test environments. > > The simplified API would be the equivalent of ignore::DeprecationWarning, > so with some additional documentation even versions of applications running > on versions of Python < 3.7 would still have an “out”. (Yes, the > simplified API is just a convenience moving forward.) > > Cheers, > -Barry > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: go back to enabling DeprecationWarning by default
On 8 November 2017 at 18:56, Barry Warsaw wrote: > On Nov 8, 2017, at 08:47, Guido van Rossum wrote: >> >> You seem to have missed Nick's posts where he clearly accepts that a middle >> ground is necessary. R D Murray is also still unconvinced. (And obviously I >> myself am against reverting to the behavior from 7 years ago.) If we can't >> agree on some middle ground, the status quo will be maintained. > > I haven’t seen a response to my suggestion, so it’s possible that it got > missed in the flurry. With coordination with setuptools, we could: > > * Re-enable DeprecationWarning by default > * Add a simplified API for specifically silencing DeprecationWarnings > * Modify setuptools to call this API for generated entry point scripts > > I think this would mean that most application users would still not see the > warnings. The simplified API would be available for handcrafted scripts to > call to accomplish the same thing the setuptools enhancement would provide. > Developers would see DeprecationWarnings in their development and test > environments. > > The simplified API would be the equivalent of ignore::DeprecationWarning, so > with some additional documentation even versions of applications running on > versions of Python < 3.7 would still have an “out”. (Yes, the simplified API > is just a convenience moving forward.) pip uses distutils for its script wrappers, but uses its own script template, so it'd need a pip change too (which means it'd be in pip 10 but not earlier versions). Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Enabling depreciation warnings feature code cutoff
On 8 November 2017 at 19:21, Antoine Pitrou wrote: > On Wed, 8 Nov 2017 11:35:13 +1000 > Nick Coghlan wrote: > >> On 8 November 2017 at 10:03, Guido van Rossum wrote: >> > OK, so let's come up with a set of heuristics that does the right thing for >> > those cases specifically. I'd say whenever you're executing code from a >> > zipfile or some such it's not considered your own code (by default). >> >> My current preferred heuristic is just to add a new default filter to the >> list: >> >> once::DeprecationWarning:__main__ > > Special cases are not special enough to break the rules. > > In other words, I'm -1 on this. Not only does it add complication and > inconsistency (bound to catch people by surprise) to an already > non-trivial set of default warning settings, but it doesn't even solve > any problem that I'm aware of. The idea that __main__ scripts should > get special treatment here is entirely gratuitous. For interactive use, the principle ends up being "Code you write gives deprecation warnings, code you import doesn't" (which is the main aspect I care about, since it's the one that semi-regularly trips me up when I forget that DeprecationWarning is off by default). Beyond that, it encourages folks distributing applications for others to use to keep __main__ simple and import most of their functionality from other libraries, which I also consider a desirable outcome. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: go back to enabling DeprecationWarning by default
On Nov 8, 2017, at 12:02, Guido van Rossum wrote: > > I hadn't seen that, but it requires too much cooperation of library owners. Actually, mostly just setuptools and as Paul points out, pip. Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Enabling depreciation warnings feature code cutoff
On Wed, Nov 8, 2017 at 10:33 PM, Nick Coghlan wrote: > For interactive use, the principle ends up being "Code you write gives > deprecation warnings, code you import doesn't" (which is the main > aspect I care about, since it's the one that semi-regularly trips me > up when I forget that DeprecationWarning is off by default). I with Antoine here. The idea that "code in __main__" is the set of code someone wrote really seems a lot like guessing (and not even very good guessing). If everyone follows the "keep __main__ small" then scripts won't automatically display deprecation warnings by default and so the original problem of "warnings are easy to miss" remains. Counter proposal -- why don't testing frameworks turn on warnings by default? E.g. like pytest-warnings? That way people running tests will see warnings and others won't unless they ask to see them. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Enabling depreciation warnings feature code cutoff
On 9 November 2017 at 07:09, Simon Cross wrote: > On Wed, Nov 8, 2017 at 10:33 PM, Nick Coghlan wrote: >> For interactive use, the principle ends up being "Code you write gives >> deprecation warnings, code you import doesn't" (which is the main >> aspect I care about, since it's the one that semi-regularly trips me >> up when I forget that DeprecationWarning is off by default). > > I with Antoine here. The idea that "code in __main__" is the set of code > someone > wrote really seems a lot like guessing (and not even very good guessing). > > If everyone follows the "keep __main__ small" then scripts won't automatically > display deprecation warnings by default and so the original problem of > "warnings > are easy to miss" remains. That's an intended outcome - the goal is to have applications that are packaged in any way (installed Python package, importable packages with a __main__ submodule, zip archives with a __main__.py file) continue to silence deprecation warnings by default, as in those cases, the folks running them are *users* of those applications, rather than developers on them. > > Counter proposal -- why don't testing frameworks turn on warnings by default? > E.g. like pytest-warnings? That way people running tests will see warnings and > others won't unless they ask to see them. They do. The problem is that this only works for things that actually have test suites, which misses a lot of "Interactive Python is part of the user experience" use cases, like CPython's own REPL, and personal scripting on a platform like Linux or conda. However, between them, the following two guidelines should provide pretty good deprecation warning coverage for the world's Python code: 1. If it's in __main__, it will emit deprecation warnings at runtime 2. If it's not in __main__, it should have a test suite Thus the future answer to "I don't want deprecation warnings at runtime" becomes "Move it out of __main__ into an importable module, and give it a test suite". Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Enabling depreciation warnings feature code cutoff
Le 08/11/2017 à 22:43, Nick Coghlan a écrit : > > However, between them, the following two guidelines should provide > pretty good deprecation warning coverage for the world's Python code: > > 1. If it's in __main__, it will emit deprecation warnings at runtime > 2. If it's not in __main__, it should have a test suite Nick, have you actually read the discussion and the complaints people had with the current situation? Most of them *don't* specifically talk about __main__ scripts. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 484: difference between tuple and ()
This is my first post on this list, so please don't kill me if I ask it in the wrong place, or if the question is stupid. I asked this question on Stack Overflow already: https://stackoverflow.com/questions/47163048/python-annotations-difference-between-tuple-and In very short, which form is correct ? |deffunc()->Tuple[int,int] ||| || But this requires to import the typing module. Or this (doesn't crash): |deffunc()->(int,int): | || || ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 484: difference between tuple and ()
2017-11-08 14:01 GMT-08:00 Jean-Patrick Francoia < jeanpatrick.franc...@gmail.com>: > This is my first post on this list, so please don't kill me if I ask it in > the wrong place, or if the question is stupid. > > > I asked this question on Stack Overflow already: > > https://stackoverflow.com/questions/47163048/python- > annotations-difference-between-tuple-and > > > In very short, which form is correct ? > > > def func() -> Tuple[int, int] > > > > But this requires to import the typing module. > > > Or this (doesn't crash): > > > def func() -> (int, int): > > > The former is correct. Type checkers should reject the second one. But because type checking in Python is through static analysis, either will work at runtime—you need to run a separate static analysis tool like mypy or pytype to find type errors in your code. Also, python-dev is a mailing list for the development of Python, not for questions about Python. The Gitter chatroom at https://gitter.im/python/typing and the typing issue tracker at https://github.com/python/typing are better places for questions about typing in Python. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > jelle.zijlstra%40gmail.com > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Enabling depreciation warnings feature code cutoff
On 9 November 2017 at 07:46, Antoine Pitrou wrote: > > Le 08/11/2017 à 22:43, Nick Coghlan a écrit : >> >> However, between them, the following two guidelines should provide >> pretty good deprecation warning coverage for the world's Python code: >> >> 1. If it's in __main__, it will emit deprecation warnings at runtime >> 2. If it's not in __main__, it should have a test suite > > Nick, have you actually read the discussion and the complaints people > had with the current situation? Most of them *don't* specifically talk > about __main__ scripts. I have, and I've also re-read the discussions regarding why the default got changed in the first place. Behaviour up until 2.6 & 3.1: once::DeprecationWarning Behaviour since 2.7 & 3.2: ignore::DeprecationWarning With test runners overriding the default filters to set it back to "once::DeprecationWarning". The rationale for that change was so that end users of applications that merely happened to be written in Python wouldn't see deprecation warnings when Linux distros (or the end user) updated to a new Python version. It had the downside that you had to remember to opt-in to deprecation warnings in order to see them, which is a problem if you mostly use Python for ad hoc personal scripting. Proposed behaviour for Python 3.7+: once::DeprecationWarning:__main__ ignore::DeprecationWarning With test runners still overriding the default filters to set them back to "once::DeprecationWarning". This is a partial reversion back to the pre-2.7 behaviour, focused specifically on interactive use and ad hoc personal scripting. For ad hoc *distributed* scripting, the changed default encourages upgrading from single-file scripts to the zipapp model, and then minimising the amount of code that runs directly in __main__.py. I expect this will be a sufficient change to solve the specific problem I'm personally concerned by, so I'm no longer inclined to argue for anything more complicated. Other folks may have other concerns that this tweak to the default filters doesn't address - they can continue to build their case for more complex options using this as the new baseline behaviour. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations
On 8 November 2017 at 16:24, Guido van Rossum wrote: > I also don't like the idea that there's nothing you can do with a thunk > besides calling it -- you can't meaningfully introspect it (not without > building your own bytecode interpreter anyway). Wait, that wasn't what I was suggesting at all - with thunks exposing their code object the same way a function does (i.e. as a `__code__` attribute), the introspection functions in `dis` would still work on them, so you'd be able to look at things like which variable names they referenced, thus granting the caller complete control over *how* they resolved those variable names (by setting them in the local namespace passed to the call). This is why they'd have interesting potential future use cases as general purpose callbacks - every local, nonlocal, global, and builtin name reference would implicitly be an optional parameter (or a required parameter if the name couldn't be resolved as a nonlocal, global, or builtin). > Using an AST instead of a string is also undesirable -- the AST changes in > each release, and the usual strong compatibility guarantees don't apply > here. And how are you going to do anything with it? If you've got a string > and you want an AST node, it's one call away. But if you've got an AST node > and you want either a string *or* the object to which that string would > evaluate, you've got a lot of work to do. Plus the AST takes up a lot more > space than the string, and we don't have a way to put an AST in a bytecode > file. (And as Inada-san pointed out a thunk *also* takes up more space than > a string.) > > Nick, please don't try to save the thunk proposal by carefully dissecting > every one of my objections. That will just prolong its demise. Just the one objection, since you seem to be rejecting something I didn't suggest (i.e. adding an opaque callable type that the dis and inspect modules didn't understand). I agree that would be a bad idea, but it's also not what I was suggesting we do. Instead, thunks would offer all the same introspection features as lambda expressions do, they'd just differ in the following ways: * the parameter list on their code objects would always be empty * the parameter list for their __call__ method would always be "ns=None" * they'd be compiled without CO_OPTIMIZED (the same as a class namespace) * they'd look up their closure references using LOAD_CLASSDEREF (the same as a class namespace) That said, even without a full-fledged thunk based solution to handling lexical scoping I think there's a way to resolve the nested class problem in PEP 563 that works for both explicitly and implicitly quoted strings, while still leaving the door open to replacing implicitly quoted strings with thunks at a later date: stating that *if* users want such nested references to be resolvable at runtime, they need to inject a runtime reference to the outermost class into the inner class namespace. That is, if you want to take: class C: field = 1 class D: def method(a: C.field): ... and move it inside a function, that would actually look like: def f(): class C: field = 1 class D: def method(a: C.field): ... C.D.C = C # Make annotations work at runtime return f That leaves the door open to a future PEP that proposes thunk-based annotations as part of proposing thunks as a new low level delayed evaluation primitive. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OrderedDict(kwargs) optimization?
> That'd be great for preserving kwargs' order after a pop() or a del? To clarify, order is preserved after pop in Python 3.6 (and maybe 3.7). There is discussion about breaking it to optimize for limited use cases, but I don't think it's worth enough to discuss more until it demonstrates real performance gain. > Is there an opportunity to support a fast cast to OrderedDict from 3.6 dict? > Can it just copy .keys() into the OrderedDict linked list?Or is there more > overhead to the transition? https://bugs.python.org/issue31265 Regards, INADA Naoki ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Clarifying Cygwin support in CPython
On 9 November 2017 at 02:28, Zachary Ware wrote: > On Wed, Nov 8, 2017 at 8:39 AM, Erik Bray wrote: >> a platform--in particular it's not clear when a buildbot is considered >> "stable", or how to achieve that without getting necessary fixes >> merged into the main branch in the first place. > > I think in this context, "stable" just means "keeps a connection to > the buildbot master and doesn't blow up when told to build" :). As > such, I'm ready to get you added to the fleet whenever you are. Yep, and from a PEP 11 perspective, it's fine to have the order be: 1. add the new buildbot worker while it's still red 2. make the changes needed to turn it green in 3.X 3. keep it green for future releases of 3.X+ There just needs to be enough lead time between step 1 and the release of 3.Xb1 to land the compatibility fixes without causing problems for the release manager (and you're currently fine on that front with respect to 3.7). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add Py_SETREF and Py_XSETREF to the stable C API
> On Nov 8, 2017, at 8:30 AM, Serhiy Storchaka wrote: > > Macros Py_SETREF and Py_XSETREF were introduced in 3.6 and backported to all > maintained versions ([1] and [2]). Despite their names they are private. I > think that they are enough stable now and would be helpful in third-party > code. Are there any objections against adding them to the stable C API? [3] I have mixed feeling about this. You and Victor seem to really like these macros, but they have been problematic for me. I'm not sure whether it is a conceptual issue or a naming issue, but the presence of these macros impairs my ability to read code and determine whether the refcounts are correct. I usually end-up replacing the code with the unrolled macro so that I can count the refs across all the code paths. The other issue is that when there are multiple occurrences of these macros for multiple variables, it interferes with my best practice of deferring all decrefs until the data structures are in a fully consistent state. Any one of these can cause arbitrary code to run. I greatly prefer putting all the decrefs at the end to increase my confidence that it is okay to run other code that might reenter the current code. Pure python functions effectively have this built-in because the locals all get decreffed at the end of the function when a return-statement is encountered. That practice helps me avoid hard to spot re-entrancy issues. Lastly, I think we should have a preference to not grow the stable C API. Bigger APIs are harder to learn and remember, not so much for you and Victor who use these frequently, but for everyone else who has to lookup all the macros whose function isn't immediately self-evident. Raymond ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] Enabling depreciation warnings feature code cutoff
On Nov 8, 2017, at 16:10, Nick Coghlan wrote: > The rationale for that change was so that end users of applications > that merely happened to be written in Python wouldn't see deprecation > warnings when Linux distros (or the end user) updated to a new Python > version. Instead they’d see breakage as DeprecationWarnings turned into errors. :( I’m not saying that Python apps, regardless of who they are provided by, should expose DeprecationWarnings to their end users. I actually think it’s good that they don’t because I don’t think most users care if their apps are written in Python, and wouldn’t know what to do about those warnings anyway. And they do cause anxiety. I suppose there are lots of ways to do this, but at least I’m pretty sure we all agree that end users shouldn’t see DeprecationWarnings, while developers should. -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal: go back to enabling DeprecationWarning by default
On 9 November 2017 at 04:56, Barry Warsaw wrote: > On Nov 8, 2017, at 08:47, Guido van Rossum wrote: >> >> You seem to have missed Nick's posts where he clearly accepts that a middle >> ground is necessary. R D Murray is also still unconvinced. (And obviously I >> myself am against reverting to the behavior from 7 years ago.) If we can't >> agree on some middle ground, the status quo will be maintained. > > I haven’t seen a response to my suggestion, so it’s possible that it got > missed in the flurry. With coordination with setuptools, we could: > > * Re-enable DeprecationWarning by default > * Add a simplified API for specifically silencing DeprecationWarnings > * Modify setuptools to call this API for generated entry point scripts > > I think this would mean that most application users would still not see the > warnings. The simplified API would be available for handcrafted scripts to > call to accomplish the same thing the setuptools enhancement would provide. > Developers would see DeprecationWarnings in their development and test > environments. > > The simplified API would be the equivalent of ignore::DeprecationWarning, so > with some additional documentation even versions of applications running on > versions of Python < 3.7 would still have an “out”. (Yes, the simplified API > is just a convenience moving forward.) I did see that, but I think a "once::DeprecationWarning:__main__" filter provides a comparable benefit in a simpler way, as the recommended idiom to turn off deprecation warnings at runtime becomes: from elsewhere import main if __name__ == "__main__": import sys sys.exit(main(sys.argv)) That same idiom will then work for: * entry point wrapper scripts * __main__ submodules in executable packages * __main__.py files in executable directories and zip archives And passing "-Wd" will still correctly override the default filter set. It doesn't resolve the problem Nathaniel pointed out that "stacklevel" can be hard to set correctly when emitting a warning (especially at import time), but it also opens a new way of dealing with that: using warnings.warn_explicit to claim that the reporting module is "__main__" if you want to increase the chances of the warning being seen by default. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations
On Wed, Nov 8, 2017 at 5:49 PM, Nick Coghlan wrote: > On 8 November 2017 at 16:24, Guido van Rossum wrote: > > I also don't like the idea that there's nothing you can do with a thunk > > besides calling it -- you can't meaningfully introspect it (not without > > building your own bytecode interpreter anyway). > > Wait, that wasn't what I was suggesting at all - with thunks exposing > their code object the same way a function does (i.e. as a `__code__` > attribute), the introspection functions in `dis` would still work on > them, so you'd be able to look at things like which variable names > they referenced, thus granting the caller complete control over *how* > they resolved those variable names (by setting them in the local > namespace passed to the call). > I understood that they would be translated to `lambda: `. It seems you have a slightly more complex idea but if you're suggesting introspection through dis, that's too complicated for my taste. This is why they'd have interesting potential future use cases as > general purpose callbacks - every local, nonlocal, global, and builtin > name reference would implicitly be an optional parameter (or a > required parameter if the name couldn't be resolved as a nonlocal, > global, or builtin). > Yeah, but that's scope creep for PEP 563. Łukasz and I are interested in gradually restricting the use of annotations to static typing with an optional runtime component. We're not interested in adding different use cases. (We're committed to backwards compatibility, but only until 4.0, with a clear deprecation path.) > > Using an AST instead of a string is also undesirable -- the AST changes > in > > each release, and the usual strong compatibility guarantees don't apply > > here. And how are you going to do anything with it? If you've got a > string > > and you want an AST node, it's one call away. But if you've got an AST > node > > and you want either a string *or* the object to which that string would > > evaluate, you've got a lot of work to do. Plus the AST takes up a lot > more > > space than the string, and we don't have a way to put an AST in a > bytecode > > file. (And as Inada-san pointed out a thunk *also* takes up more space > than > > a string.) > > > > Nick, please don't try to save the thunk proposal by carefully dissecting > > every one of my objections. That will just prolong its demise. > > Just the one objection, since you seem to be rejecting something I > didn't suggest (i.e. adding an opaque callable type that the dis and > inspect modules didn't understand). I agree that would be a bad idea, > but it's also not what I was suggesting we do. > I did not assume totally opaque -- but code objects are not very introspection friendly (and they have no strong compatibility guarantees). Instead, thunks would offer all the same introspection features as > lambda expressions do, they'd just differ in the following ways: > > * the parameter list on their code objects would always be empty > * the parameter list for their __call__ method would always be "ns=None" > * they'd be compiled without CO_OPTIMIZED (the same as a class namespace) > * they'd look up their closure references using LOAD_CLASSDEREF (the > same as a class namespace) > I don't understand the __call__ with "ns-None" thing but I don't expect it matters. > That said, even without a full-fledged thunk based solution to > handling lexical scoping I think there's a way to resolve the nested > class problem in PEP 563 that works for both explicitly and implicitly > quoted strings, while still leaving the door open to replacing > implicitly quoted strings with thunks at a later date: stating that > *if* users want such nested references to be resolvable at runtime, > they need to inject a runtime reference to the outermost class into > the inner class namespace. > > That is, if you want to take: > > class C: > field = 1 > class D: > def method(a: C.field): > ... > > and move it inside a function, that would actually look like: > > def f(): > class C: > field = 1 > class D: > def method(a: C.field): > ... > C.D.C = C # Make annotations work at runtime > return f > > That leaves the door open to a future PEP that proposes thunk-based > annotations as part of proposing thunks as a new low level delayed > evaluation primitive. > Sorry, that's not a door I'd like to leave open. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
