Hi, tl;dr Tests of the Python Test Suite now start with 131 imported modules, instead of 233. For example, asyncio, logging, multiprocessing and warnings are no longer imported by default.
-- The Python test suite is run by a custom test runner called "libregrtest" (test.libregrtest). It can detect reference leaks, write the output as JUnit XML, run tests in parallel with multiple processes, etc. Tests are written with test.support which is a collection of helper functions. Over the years, test.support got more and more functions, and libregrtest got more and more features. The problem is that they import more and more modules. For example, "import test.support" imports 173 modules in Python 3.8! $ python3.8 >>> import sys, sys >>> before=set(sys.modules); import test.support; after=set(sys.modules) >>> len(after - before) 173 In these imports, you can find some "heavy" modules like asyncio and multiprocessing. Moreover, some modules have "side effects" on import. For example, "import logging" registers an "at fork" callback. In April 2020, I worked with Hai Shi on Python 3.10 in bpo-40275 to reduce the number of test.support imports from 171 to 25! test.support was splitted into multiple sub-modules: - bytecode_helper - hashlib_helper - import_helper - logging_helper - os_helper - script_helper - socket_helper - threading_helper - warnings_helper I continued the work in bpo-41718 to reduce the number of libregrtest imports, since libregrtest also imported many modules. A dummy test which does nothing in the master branch now only has 131 modules in sys.modules, whereas in Python 3.9 it has 233 (+102) modules! For example, asyncio, logging, multiprocessing and warnings are no longer imported by default. "import test.support" is now faster. master branch compared to Python 3.8 using ./python -c 'import test.support' command and Python built in release mode: [py38] 85.7 ms +- 6.8 ms -> [master] 32.4 ms +- 1.3 ms: 2.64x faster More realistic command running a single test method: ./python -m test test_os -m test_access [py38] 136 ms +- 5 ms -> [master] 97.8 ms +- 2.4 ms: 1.39x faster The only annoying point is that splitting test.support into sub-modules was not backported to Python 3.8 and 3.9, and so it will make backports a little bit more annoying. Sorry about that! -- If you know that two modules should be tested together, please write a dedicated test for that ;-) Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/I3OQTA3F66NQUN7CH2NHC5XZTO24QCIK/ Code of Conduct: http://python.org/psf/codeofconduct/
