[Python-Dev] Re: Remove asyncore, asynchat and smtpd modules
On 16. 11. 21 1:36, Victor Stinner wrote: As I wrote previously, the DeprecationWarning warning is only emitted at runtime since Python 3.10. Since my PR got 5 approvals, I just merged it: https://github.com/python/cpython/pull/29521 No mater the number of approvals, this removal does not follow the policy. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RVSMVPGFWCTFDGHKHBZ72NMXXFT2EIIU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2021-11-26 - 2021-12-03)
Python tracker at https://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open7252 (-14)
closed 50449 (+82)
total 57701 (+68)
Open issues with patches: 2884
Issues opened (47)
==
#44530: Propagate qualname from the compiler unit to code objects for
https://bugs.python.org/issue44530 reopened by steve.dower
#45813: Importing asyncio after deleting a coroutine object and before
https://bugs.python.org/issue45813 reopened by Dennis Sweeney
#45906: Python github installation issue
https://bugs.python.org/issue45906 opened by mazen001.ahmed001
#45909: sysconfig --generate-posix-vars creates wrong file when cross
https://bugs.python.org/issue45909 opened by christian.heimes
#45910: mailbox should support options for calling email parser
https://bugs.python.org/issue45910 opened by bpoaugust
#45913: readline + GTK + Pytest Seg Fault with Python 3.7 and 3.10 on
https://bugs.python.org/issue45913 opened by danyeaw
#45914: Very first multiprocessing example not working on Windows 11
https://bugs.python.org/issue45914 opened by quattrozhou
#45915: Use fcntl(fd, F_GETFD) to check whether an fd is valid
https://bugs.python.org/issue45915 opened by christian.heimes
#45916: documentation link error
https://bugs.python.org/issue45916 opened by cookiez6
#45919: Use WinAPI GetFileType() in is_valid_fd()
https://bugs.python.org/issue45919 opened by eryksun
#45921: codecs module doesn't support iso-8859-6-i, iso-8859-6-e, iso-
https://bugs.python.org/issue45921 opened by msapiro
#45922: Many method, function, built-in... are not clickable and shoul
https://bugs.python.org/issue45922 opened by Arthur-Milchior
#45923: Improve performance of sys.settracing based tools.
https://bugs.python.org/issue45923 opened by Mark.Shannon
#45924: Incorrect traceback when future's exception is raised multiple
https://bugs.python.org/issue45924 opened by iritkatriel
#45925: Upgrade macOS and Windows installers to use SQLite 3.37.0
https://bugs.python.org/issue45925 opened by erlendaasland
#45926: singledispatchmethod doesn't handle named arguments
https://bugs.python.org/issue45926 opened by alisson276
#45927: timeit accepts -c/--clock and -t/--time without any functional
https://bugs.python.org/issue45927 opened by lemburg
#45929: extend json.tool --json-lines to ignore empty rows
https://bugs.python.org/issue45929 opened by ZeD
#45934: python curses newterm implementation
https://bugs.python.org/issue45934 opened by draganic1
#45935: Add test for Issue11109: socketserver.ForkingMixIn leaves zomb
https://bugs.python.org/issue45935 opened by iritkatriel
#45937: Pdb can't use the commands through -c or .pdbrc files
https://bugs.python.org/issue45937 opened by Zrincet
#45938: EmailMessage as_bytes
https://bugs.python.org/issue45938 opened by marc.villain
#45940: add_multiarch_paths breaks cross compilation to Emscripten
https://bugs.python.org/issue45940 opened by ethan smith
#45942: shutil.copytree can raise OSErrors not wrapped in shutil.Error
https://bugs.python.org/issue45942 opened by Floozutter
#45944: Avoid calling isatty() for most open() calls
https://bugs.python.org/issue45944 opened by collinanderson
#45945: compileall.py throws a traceback when using -j0 and thus 'make
https://bugs.python.org/issue45945 opened by Alexander Kanavin
#45946: RecursionError when annotating a field with the same name as a
https://bugs.python.org/issue45946 opened by Gobot1234
#45947: Place dict (and values) pointers at a fixed (negative) offset
https://bugs.python.org/issue45947 opened by Mark.Shannon
#45948: Unexpected instantiation behavior for xml.etree.ElementTree.XM
https://bugs.python.org/issue45948 opened by rdsteed
#45949: Provide pure-Python implementation of Programs/_freeze_module
https://bugs.python.org/issue45949 opened by christian.heimes
#45950: Reintroduce bootstrap_python for freezing
https://bugs.python.org/issue45950 opened by christian.heimes
#45953: Statically allocate interpreter states as much as possible.
https://bugs.python.org/issue45953 opened by eric.snow
#45955: Calling read() on HTTPError may cause KeyError in tempfile
https://bugs.python.org/issue45955 opened by sivel
#45957: _tkinter.TclError: expected boolean value but got ""
https://bugs.python.org/issue45957 opened by amin-nejad
#45959: Teach pprint about dict views
https://bugs.python.org/issue45959 opened by rhettinger
#45960: bullseye arm/v7 time.time() Operation not permitted
https://bugs.python.org/issue45960 opened by SaschaJohn
#45962: Clarify that PyModule_AddString{Constant,Macro} use utf-8
https://bugs.python.org/issue45962 opened by Antony.Lee
#45963: Embed interpreter frame in generator.
https://bugs.python.org/issue45963 opened by Mark.Shannon
#45964: gdb test fails when packaging Python 3.10 for
[Python-Dev] Re: Tool to search in the source code of PyPI top 5000 projects
Hi Steve, I completely agree with all you said ;-) I will not debate here if incompatible changes are worth it or not, this topic was discussed recently in another thread. On Fri, Dec 3, 2021 at 2:56 PM Steve Dower wrote: > FTR, I don't consider the top projects on PyPI to be representative of > our user base, and *especially* not representative of people compiling > native modules. > > This is not a good way to evaluate the impact of breaking changes. I do not pretend that a code search on PyPI top 5000 projects is only way and is an exhaustive way to measure the impact of incompatible changes. I'm only trying to advertize that *there is one practical tool* which is better than nothing. Last years, I saw many incompatible changes introduced in Python without: * estimating how many projects: "release Python and pray" in the hope that only a minority is impacted * don't document the change at all, or just say that it's now broken, but it was rare that practical instructions were provided to explain how to port code and how to keep support for old Python versions. I saw a net enhancement recently. Better documentation, core devs proactive to fix impacted projects, better communication to announce incompatible changes in advance, and practical instructions to port code without losing support for old Python versions. > It would be far safer to assume that every change is going to break > someone and evaluate: > * how will they find out that upgrading Python will cause them to break > * how will they find out where that break occurs > * how will they find out how to fix it > * how will they manage that fix across multiple releases > * how will we explain that upgrading and fixing breaks is better for > *them* than staying on the older version In the PEP 674, I wrote an explicit section "Port C extensions to Python 3.11": https://www.python.org/dev/peps/pep-0674/#port-c-extensions-to-python-3-11 It doesn't cover all your questions, but it tries to reply to most of them. I'm open to suggestions to enhance this section ;-) IMO it's a good practice that a PEP introducing incompatible changes explains how to port existing code and this practice should become more common ;-) > * how will we explain that upgrading and fixing breaks is better for > *them* than staying on the older version This part is always the hardest :-( Staying at an old Python version is usually cheaper: no further developments needed. There are still companies using Python 2 nowadays. Don't underestimate the technical debt and the cost to upgrade ;-) For the PEP 674, the promise is that updated C extensions should work better with HPy and GraalPython. Not sure if it's enough to motivate developers to port their code. IMO one important thing is the cost of upgrading a C extension. For the PEP 674, all you need to do is to run a single command once! => ./upgrade_pythoncapi.py path/to/project/ Done! It reminds me the Python 2 to Python 3 migration before 2to3 and six were usable and popular. The migration was super painful and so nobody wanted to do it because everybody wanted to still keep support for Python 2. Only adding Python 3 support didn't bring any benefit in the short term (Python 3 only features couldn't be used). People didn't migrate because migrating code was dangerous, painful and complicated. I'm now in favor of limiting the number of incompatible changes per Python release and never again do a Python 4 "break the world" release. I prefer to have a bunch of incompatible changes in each Python release :-) > This last one is particularly important, as many large organisations > (anecdotally) seem to have settled on Python 3.7 for a while now. > Inevitably, this means they're all going to be faced with a painful time > when it comes to an upgrade, and every little change we add on is going > to hurt more. Every extra thing that needs fixing is motivation to just > rewrite in a new language with more hype (and the promise of better > compatibility... which I won't comment specifically on, but I suspect > they won't manage it any better than us ;) ). IMO we need to invest more time on developing tools to ease the migration to newer Python versions, like: * Python: https://github.com/asottile/pyupgrade * C code: https://github.com/pythoncapi/pythoncapi_compat Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RWJQM5AQGQLM46TJSQZN7CBXVYS5BFPD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Tool to search in the source code of PyPI top 5000 projects
FTR, I don't consider the top projects on PyPI to be representative of
our user base, and *especially* not representative of people compiling
native modules.
This is not a good way to evaluate the impact of breaking changes.
It would be far safer to assume that every change is going to break
someone and evaluate:
* how will they find out that upgrading Python will cause them to break
* how will they find out where that break occurs
* how will they find out how to fix it
* how will they manage that fix across multiple releases
* how will we explain that upgrading and fixing breaks is better for
*them* than staying on the older version
This last one is particularly important, as many large organisations
(anecdotally) seem to have settled on Python 3.7 for a while now.
Inevitably, this means they're all going to be faced with a painful time
when it comes to an upgrade, and every little change we add on is going
to hurt more. Every extra thing that needs fixing is motivation to just
rewrite in a new language with more hype (and the promise of better
compatibility... which I won't comment specifically on, but I suspect
they won't manage it any better than us ;) ).
This is not the case for the top PyPI projects. They incrementally
update and crowdsource fixes, often from us. The pain is distributed to
the level of permanent background noise, which sucks in its own way, but
is ultimately not representative of much of our user base.
So by all means, use this tool for checking stuff. But it's not a
substitute for justifying every incompatible change in its own right.
/rant
Cheers,
Steve
On 12/2/2021 11:44 PM, Victor Stinner wrote:
Hi,
I wrote two scripts based on the work of INADA-san's work to (1)
download the source code of the PyPI top 5000 projects (2) search for
a regex in these projects (compressed source archives).
You can use these tools if you work on an incompatible Python or C API
change to estimate how many projects are impacted.
The HPy project created a Git repository for a similar need (latest
update in June 2021):
https://github.com/hpyproject/top4000-pypi-packages
There are also online services for code search:
* GitHub: https://github.com/search
* https://grep.app/ (I didn't try it yet)
* Debian: https://codesearch.debian.net/
(1) Dowload
Script:
https://github.com/vstinner/misc/blob/main/cpython/download_pypi_top.py
Usage: download_pypi_top.py PATH
It uses this JSON file:
https://hugovk.github.io/top-pypi-packages/top-pypi-packages-30-days.min.json
From this service:
https://hugovk.github.io/top-pypi-packages/
At December 1, on 5000 projects, it only downloads 4760 tarball and
ZIP archives: I guess that 240 projects don't provide a source
archive. It takes around 5,2 GB of disk space.
(2) Code search
First, I used the fast and nice "ripgrep" tool with the command "rg
-zl REGEX path/*.{zip,gz,bz2,tgz}" (-z searchs in ZIP and tarball
archives). But it doesn't show the path inside the archive and it
searchs in files generated by Cython whereas I wanted to ignore these
files.
So I wrote a short Python script which decompress tarball and ZIP
archive in memory and looks for a regex:
https://github.com/vstinner/misc/blob/main/cpython/search_pypi_top.py
Usage: search_pypi_top.py "REGEX" output_filename
The code to parse command line option is hardcoded and pypi_dir =
"PYPI-2021-12-01-TOP-5000" are hardcoded :-D
It ignores files generated by Cython and .so binary files (Linux
dynamic libraries).
While "rg" is very fast, my script is very slow. But I don't care,
once the regex is written, I only need to search for the regex once, I
can wait 10-15 min ;-) I prefer to wait longer and have a more
accurate result. Also, there is room for enhancement, like running
multiple jobs in different processes or threads.
Victor
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/HO7PS57UCJPJLON2BJPPEBM7I3Q6AM2U/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Tool to search in the source code of PyPI top 5000 projects
Hi, You're correct that the download_pypi_top.py script only downloads the latest version. I'm looking for projects impacted by incompatible changes. If the latest version is fine, a project just has to update its dependencies. If the latest version has an issue, it's very likely that old versions are also affected. Victor On Fri, Dec 3, 2021 at 8:35 AM Michał Górny wrote: > > On Fri, 2021-12-03 at 00:44 +0100, Victor Stinner wrote: > > I wrote two scripts based on the work of INADA-san's work to (1) > > download the source code of the PyPI top 5000 projects (2) search for > > a regex in these projects (compressed source archives). > > > > You can use these tools if you work on an incompatible Python or C API > > change to estimate how many projects are impacted. > > > > Am I correct that this script downloads only the newest version for each > package? It might be worth to add a disclaimer that since many Python > packages pin their dependencies to old versions, you are quite likely to > miss impact on projects that are using the deprecated API in old > versions that are still used because of their reverse dependencies. > > -- > Best regards, > Michał Górny > -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6RZML4FAVDKL6A5TKPFGQJZ4FEGKHGV6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Oh look, I've been subscribed to python/issues-test-2 notifications again
Hi Victor, On Thu, Dec 2, 2021 at 11:48 AM Victor Stinner wrote: > > Hi Ezio, > > What is the status of migrating Python issues to GitHub? You can check the status here: https://github.com/psf/gh-migration/projects/1 > Is it done? Not yet, but we are aiming for mid-January. > If not, what are remaining issues? See https://github.com/psf/gh-migration/issues There are a number of ancillary tools that need to be replaced that I haven't looked into yet, and any help with those would be appreciated. These are not blockers for the migration though, and they might be addressed after the migration happened. --Ezio > > Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/I254N235RMFVF5PYFOTPQSU7LCMYDDIH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: The current state of typing PEPs
On Fri, 3 Dec 2021 at 00:10, Shantanu Jain wrote: > > @Paul > > > ... missing resource is a central set of typing documentation that includes > > examples, FAQs and best practices as well as reference materials > > Like Sebastian, I agree, and this is something we're making progress on. That's great to know - it's easy for me to say "we need more docs" but I know it's hard to produce them. I'm glad it's on the radar :-) > > ... easy way of testing that the stubs are correct > > mypy ships with a tool called stubtest. It's what typeshed uses to validate > itself against CPython. I recently wrote up documentation for it, see here: > https://mypy.readthedocs.io/en/latest/stubtest.html Thanks, I wasn't aware of that. Although TBH I like the suggestion of using typeshed as a kind of staging point for interested users to develop annotations, which can then be moved back into the library once they've been proven stable. I'm still surprised that mypy doesn't check stub files though. If I write stubs for my external API, and then later decide I want to add types to my internal functions, to do static checking on my library, am I therefore required to move the annotations out of the stub file into the main code? What if I want to avoid the runtime overhead of importing the typing module? Or I want to keep the source code clear, by not making function definitions into multi-line monsters because of type declarations? > > A lot of the frustration I see being expressed here (including my own) > > seems to come from the fact that it's so difficult to actually take that > > sort of "I can ignore it if I don't use it" > > Thanks for not staying quiet and helping us make typing better. In > particular, I think one takeaway from this thread is that a lot of our > documentation is focussed on "how to get started with typing my code", and > not much addressed at library authors with limited time to deal with typing > questions (like "what is a PEP 561" and "if my type hints are incomplete does > that cause problems for my users") That's a good point that I hadn't spotted - thanks for picking up on it! Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CRBIOVH4K5VFIIPB2D6EJCIVWJXGITEU/ Code of Conduct: http://python.org/psf/codeofconduct/
