What do you propose to make Python startup faster? As I wrote in my previous emails, many Python core developers care of the startup time and we are working on making it faster.
INADA Naoki added -X importtime to identify slow imports and understand where Python spent its startup time. Recent example: Barry Warsaw identified that pkg_resources is slow and added importlib.resources to Python 3.7: https://docs.python.org/dev/library/importlib.html#module-importlib.resources Brett Cannon is also working on a standard solution for lazy imports since many years: https://pypi.org/project/modutil/ https://snarky.ca/lazy-importing-in-python-3-7/ Nick Coghlan is working on the C API to configure Python startup: PEP 432. When it will be ready, maybe Mercurial could use a custom Python optimized for its use case. IMHO Python import system is inefficient. We try too many alternative names. Example with Python 3.8 $ ./python -vv: >>> import dontexist # trying /home/vstinner/prog/python/master/dontexist.cpython-38dm-x86_64-linux-gnu.so # trying /home/vstinner/prog/python/master/dontexist.abi3.so # trying /home/vstinner/prog/python/master/dontexist.so # trying /home/vstinner/prog/python/master/dontexist.py # trying /home/vstinner/prog/python/master/dontexist.pyc # trying /home/vstinner/prog/python/master/Lib/dontexist.cpython-38dm-x86_64-linux-gnu.so # trying /home/vstinner/prog/python/master/Lib/dontexist.abi3.so # trying /home/vstinner/prog/python/master/Lib/dontexist.so # trying /home/vstinner/prog/python/master/Lib/dontexist.py # trying /home/vstinner/prog/python/master/Lib/dontexist.pyc # trying /home/vstinner/prog/python/master/build/lib.linux-x86_64-3.8-pydebug/dontexist.cpython-38dm-x86_64-linux-gnu.so # trying /home/vstinner/prog/python/master/build/lib.linux-x86_64-3.8-pydebug/dontexist.abi3.so # trying /home/vstinner/prog/python/master/build/lib.linux-x86_64-3.8-pydebug/dontexist.so # trying /home/vstinner/prog/python/master/build/lib.linux-x86_64-3.8-pydebug/dontexist.py # trying /home/vstinner/prog/python/master/build/lib.linux-x86_64-3.8-pydebug/dontexist.pyc # trying /home/vstinner/.local/lib/python3.8/site-packages/dontexist.cpython-38dm-x86_64-linux-gnu.so # trying /home/vstinner/.local/lib/python3.8/site-packages/dontexist.abi3.so # trying /home/vstinner/.local/lib/python3.8/site-packages/dontexist.so # trying /home/vstinner/.local/lib/python3.8/site-packages/dontexist.py # trying /home/vstinner/.local/lib/python3.8/site-packages/dontexist.pyc Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked ModuleNotFoundError: No module named 'dontexist' Why do we still check for the .pyc file outside __pycache__ directories? Why do we have to check for 3 different names for .so files? Does Mercurial need all directories of sys.path? What's the status of the "system python" project? :-) I also would prefer Python without the site module. Can we rewrite this module in C maybe? Until recently, the site module was needed on Python to create the "mbcs" encoding alias. Hopefully, the feature has been removed into Lib/encodings/__init__.py (new private _alias_mbcs() function). Python 3.7b3+: $ python3.7 -X importtime -c pass import time: self [us] | cumulative | imported package import time: 95 | 95 | zipimport import time: 589 | 589 | _frozen_importlib_external import time: 67 | 67 | _codecs import time: 498 | 565 | codecs import time: 425 | 425 | encodings.aliases import time: 641 | 1629 | encodings import time: 228 | 228 | encodings.utf_8 import time: 143 | 143 | _signal import time: 335 | 335 | encodings.latin_1 import time: 58 | 58 | _abc import time: 265 | 322 | abc import time: 298 | 619 | io import time: 69 | 69 | _stat import time: 196 | 265 | stat import time: 169 | 169 | genericpath import time: 336 | 505 | posixpath import time: 1190 | 1190 | _collections_abc import time: 600 | 2557 | os import time: 223 | 223 | _sitebuiltins import time: 214 | 214 | sitecustomize import time: 74 | 74 | usercustomize import time: 477 | 3544 | site Victor _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
