Based-on: <20210625154540.783306-1-js...@redhat.com> GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-iotest CI: https://gitlab.com/jsnow/qemu/-/pipelines/327413868
Since iotests are such a heavy and prominent user of the Python qemu.qmp and qemu.machine packages, it would be convenient if the Python linting suite also checked this client for any possible regressions introduced by shifting around signatures, types, or interfaces in these packages. (We'd eventually find those problems when iotest 297 ran, but with increasing distance between Python development and Block development, the risk of an accidental breakage in this regard increases. I, personally, know to run iotests (and especially 297) after changing Python code, but not everyone in the future might. Plus, I am lazy, and I like only having to push one button.) Add the ability for the Python CI to run the iotest linters too, which means that the iotest linters would be checked against: - Python 3.6, using a frozen set of linting packages at their oldest supported versions, using 'pipenv' - Python 3.6 through Python 3.10 inclusive, using 'tox' and the latest versions of mypy/pylint that happen to be installed during test time. This CI test is allowed to fail with a warning, and can serve as a bellwether for when new incompatible changes may disrupt the linters. Testing against old and new Python interpreters alike can help surface incompatibility issues we may need to be aware of.) Here are example outputs of those CI jobs with this series applied: - "check-python-pipenv": https://gitlab.com/jsnow/qemu/-/jobs/1377735087 - "check-python-tox": https://gitlab.com/jsnow/qemu/-/jobs/1377735088 You can also run these same tests locally from ./python, plus one more: - "make check-venv" to test against whatever python you have. - "make check-pipenv", if you have Python 3.6 and pipenv installed. - "make check-tox", if you have Python 3.6 through Python 3.10 installed. Here are example outputs from each of the three different local execution methods, in order as outlined above: (1) jsnow@scv ~/s/q/python (python-package-iotest)> make check-venv VENV ~/.cache/qemu-pyvenv ACTIVATE ~/.cache/qemu-pyvenv INSTALL qemu[devel] ~/.cache/qemu-pyvenv ACTIVATE ~/.cache/qemu-pyvenv make[1]: Entering directory '/home/jsnow/src/qemu/python' JOB ID : 645eacd5ed7f4d5a76ea5f494984a55403a11081 JOB LOG : /home/jsnow/avocado/job-results/job-2021-06-25T12.01-645eacd/job.log (1/5) tests/flake8.sh: PASS (0.28 s) (2/5) tests/iotests.sh: PASS (9.61 s) (3/5) tests/isort.sh: PASS (0.09 s) (4/5) tests/mypy.sh: PASS (0.25 s) (5/5) tests/pylint.sh: PASS (4.13 s) RESULTS : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 14.73 s make[1]: Leaving directory '/home/jsnow/src/qemu/python' (2) jsnow@scv ~/s/q/python (python-package-iotest)> make check-pipenv Creating a virtualenv for this project... Pipfile: /home/jsnow/src/qemu/python/Pipfile Using /usr/bin/python3.6m (3.6.13) to create virtualenv... ⠹ Creating virtual environment...created virtual environment CPython3.6.13.final.0-64 in 104ms creator CPython3Posix(dest=/home/jsnow/src/qemu/python/.venv, clear=False, no_vcs_ignore=False, global=False) seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/jsnow/.local/share/virtualenv) added seed packages: pip==21.1.2, setuptools==57.0.0, wheel==0.36.2 activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator ✔ Successfully created virtual environment! Virtualenv location: /home/jsnow/src/qemu/python/.venv Installing dependencies from Pipfile.lock (c13e91)... 🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 30/30 — 00:00:08 To activate this project's virtualenv, run pipenv shell. Alternatively, run a command inside the virtualenv with pipenv run. All dependencies are now up-to-date! rm -f pyproject.toml make[1]: Entering directory '/home/jsnow/src/qemu/python' JOB ID : a56f53f3c5ef0058d341917228362db8ff24075f JOB LOG : /home/jsnow/avocado/job-results/job-2021-06-25T11.58-a56f53f/job.log (1/5) tests/flake8.sh: PASS (0.42 s) (2/5) tests/iotests.sh: PASS (8.89 s) (3/5) tests/isort.sh: PASS (0.24 s) (4/5) tests/mypy.sh: PASS (0.30 s) (5/5) tests/pylint.sh: PASS (3.96 s) RESULTS : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 14.16 s make[1]: Leaving directory '/home/jsnow/src/qemu/python' (3) jsnow@scv ~/s/q/python (python-package-iotest)> make check-tox GLOB sdist-make: /home/jsnow/src/qemu/python/setup.py py36 create: /home/jsnow/src/qemu/python/.tox/py36 py36 installdeps: .[devel], .[fuse] py36 inst: /home/jsnow/src/qemu/python/.tox/.tmp/package/1/qemu-0.6.1.0a1.zip py36 installed: appdirs==1.4.4,astroid==2.5.6,avocado-framework==89.0,distlib==0.3.2,filelock==3.0.12,flake8==3.9.2,fusepy==3.0.1,importlib-metadata==4.5.0,importlib-resources==5.1.4,isort==5.9.1,lazy-object-proxy==1.6.0,mccabe==0.6.1,mypy==0.910,mypy-extensions==0.4.3,packaging==20.9,pluggy==0.13.1,py==1.10.0,pycodestyle==2.7.0,pyflakes==2.3.1,pylint==2.8.3,pyparsing==2.4.7,qemu @ file:///home/jsnow/src/qemu/python/.tox/.tmp/package/1/qemu-0.6.1.0a1.zip,six==1.16.0,toml==0.10.2,tox==3.23.1,typed-ast==1.4.3,typing-extensions==3.10.0.0,virtualenv==20.4.7,wrapt==1.12.1,zipp==3.4.1 py36 run-test-pre: PYTHONHASHSEED='835116428' py36 run-test: commands[0] | make check JOB ID : c37bc201d0ec4d22a9f8b9347e3848e47fe88ba8 JOB LOG : /home/jsnow/avocado/job-results/job-2021-06-25T11.59-c37bc20/job.log (1/5) tests/flake8.sh: PASS (0.36 s) (2/5) tests/iotests.sh: PASS (10.64 s) (3/5) tests/isort.sh: PASS (0.11 s) (4/5) tests/mypy.sh: PASS (1.31 s) (5/5) tests/pylint.sh: PASS (4.45 s) RESULTS : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 17.28 s py37 create: /home/jsnow/src/qemu/python/.tox/py37 py37 installdeps: .[devel], .[fuse] py37 inst: /home/jsnow/src/qemu/python/.tox/.tmp/package/1/qemu-0.6.1.0a1.zip py37 installed: appdirs==1.4.4,astroid==2.5.6,avocado-framework==89.0,distlib==0.3.2,filelock==3.0.12,flake8==3.9.2,fusepy==3.0.1,importlib-metadata==4.5.0,isort==5.9.1,lazy-object-proxy==1.6.0,mccabe==0.6.1,mypy==0.910,mypy-extensions==0.4.3,packaging==20.9,pluggy==0.13.1,py==1.10.0,pycodestyle==2.7.0,pyflakes==2.3.1,pylint==2.8.3,pyparsing==2.4.7,qemu @ file:///home/jsnow/src/qemu/python/.tox/.tmp/package/1/qemu-0.6.1.0a1.zip,six==1.16.0,toml==0.10.2,tox==3.23.1,typed-ast==1.4.3,typing-extensions==3.10.0.0,virtualenv==20.4.7,wrapt==1.12.1,zipp==3.4.1 py37 run-test-pre: PYTHONHASHSEED='835116428' py37 run-test: commands[0] | make check JOB ID : 5fd0aaadb8557f941d773dd28f0017bed7b36526 JOB LOG : /home/jsnow/avocado/job-results/job-2021-06-25T11.59-5fd0aaa/job.log (1/5) tests/flake8.sh: PASS (0.32 s) (2/5) tests/iotests.sh: PASS (11.52 s) (3/5) tests/isort.sh: PASS (0.09 s) (4/5) tests/mypy.sh: PASS (0.27 s) (5/5) tests/pylint.sh: PASS (4.44 s) RESULTS : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 17.03 s py38 create: /home/jsnow/src/qemu/python/.tox/py38 py38 installdeps: .[devel], .[fuse] py38 inst: /home/jsnow/src/qemu/python/.tox/.tmp/package/1/qemu-0.6.1.0a1.zip py38 installed: appdirs==1.4.4,astroid==2.5.6,avocado-framework==89.0,distlib==0.3.2,filelock==3.0.12,flake8==3.9.2,fusepy==3.0.1,isort==5.9.1,lazy-object-proxy==1.6.0,mccabe==0.6.1,mypy==0.910,mypy-extensions==0.4.3,packaging==20.9,pluggy==0.13.1,py==1.10.0,pycodestyle==2.7.0,pyflakes==2.3.1,pylint==2.8.3,pyparsing==2.4.7,qemu @ file:///home/jsnow/src/qemu/python/.tox/.tmp/package/1/qemu-0.6.1.0a1.zip,six==1.16.0,toml==0.10.2,tox==3.23.1,typing-extensions==3.10.0.0,virtualenv==20.4.7,wrapt==1.12.1 py38 run-test-pre: PYTHONHASHSEED='835116428' py38 run-test: commands[0] | make check JOB ID : dde7217f6afa2cb7496b8b8da3dee57783fa6377 JOB LOG : /home/jsnow/avocado/job-results/job-2021-06-25T11.59-dde7217/job.log (1/5) tests/flake8.sh: PASS (0.25 s) (2/5) tests/iotests.sh: PASS (10.17 s) (3/5) tests/isort.sh: PASS (0.08 s) (4/5) tests/mypy.sh: PASS (0.25 s) (5/5) tests/pylint.sh: PASS (3.96 s) RESULTS : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 15.08 s py39 create: /home/jsnow/src/qemu/python/.tox/py39 py39 installdeps: .[devel], .[fuse] py39 inst: /home/jsnow/src/qemu/python/.tox/.tmp/package/1/qemu-0.6.1.0a1.zip py39 installed: appdirs==1.4.4,astroid==2.5.6,avocado-framework==89.0,distlib==0.3.2,filelock==3.0.12,flake8==3.9.2,fusepy==3.0.1,isort==5.9.1,lazy-object-proxy==1.6.0,mccabe==0.6.1,mypy==0.910,mypy-extensions==0.4.3,packaging==20.9,pluggy==0.13.1,py==1.10.0,pycodestyle==2.7.0,pyflakes==2.3.1,pylint==2.8.3,pyparsing==2.4.7,qemu @ file:///home/jsnow/src/qemu/python/.tox/.tmp/package/1/qemu-0.6.1.0a1.zip,six==1.16.0,toml==0.10.2,tox==3.23.1,typing-extensions==3.10.0.0,virtualenv==20.4.7,wrapt==1.12.1 py39 run-test-pre: PYTHONHASHSEED='835116428' py39 run-test: commands[0] | make check JOB ID : 9c30aa6af6aece73acd995603a11df6f6d609b43 JOB LOG : /home/jsnow/avocado/job-results/job-2021-06-25T12.00-9c30aa6/job.log (1/5) tests/flake8.sh: PASS (0.31 s) (2/5) tests/iotests.sh: PASS (10.97 s) (3/5) tests/isort.sh: PASS (0.09 s) (4/5) tests/mypy.sh: PASS (0.24 s) (5/5) tests/pylint.sh: PASS (4.14 s) RESULTS : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 16.11 s py310 create: /home/jsnow/src/qemu/python/.tox/py310 py310 installdeps: .[devel], .[fuse] py310 inst: /home/jsnow/src/qemu/python/.tox/.tmp/package/1/qemu-0.6.1.0a1.zip py310 installed: appdirs==1.4.4,astroid==2.5.6,avocado-framework==89.0,distlib==0.3.2,filelock==3.0.12,flake8==3.9.2,fusepy==3.0.1,isort==5.9.1,lazy-object-proxy==1.6.0,mccabe==0.6.1,mypy==0.910,mypy-extensions==0.4.3,packaging==20.9,pluggy==0.13.1,py==1.10.0,pycodestyle==2.7.0,pyflakes==2.3.1,pylint==2.8.3,pyparsing==2.4.7,qemu @ file:///home/jsnow/src/qemu/python/.tox/.tmp/package/1/qemu-0.6.1.0a1.zip,six==1.16.0,toml==0.10.2,tox==3.23.1,typing-extensions==3.10.0.0,virtualenv==20.4.7,wrapt==1.12.1 py310 run-test-pre: PYTHONHASHSEED='835116428' py310 run-test: commands[0] | make check JOB ID : 24de320dd6b9130a6986c67ea3bf9aff6af7b134 JOB LOG : /home/jsnow/avocado/job-results/job-2021-06-25T12.00-24de320/job.log (1/5) tests/flake8.sh: PASS (0.29 s) (2/5) tests/iotests.sh: PASS (18.38 s) (3/5) tests/isort.sh: PASS (0.09 s) (4/5) tests/mypy.sh: PASS (0.54 s) (5/5) tests/pylint.sh: PASS (4.10 s) RESULTS : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 23.70 s ___________________________________ summary ___________________________________ py36: commands succeeded py37: commands succeeded py38: commands succeeded py39: commands succeeded py310: commands succeeded congratulations :) John Snow (10): iotests/297: modify is_python_file to work from any CWD iotests/297: Add get_files() function iotests/297: Don't rely on distro-specific linter binaries iotests/297: Create main() function iotests/297: Separate environment setup from test execution iotests/297: Add 'directory' argument to run_linters iotests/297: return error code from run_linters() iotests/297: split linters.py off from 297 iotests/linters: Add entry point for Python CI linters python: Add iotest linters to test suite python/tests/iotests.sh | 2 + tests/qemu-iotests/297 | 80 +++------------------ tests/qemu-iotests/linters.py | 130 ++++++++++++++++++++++++++++++++++ 3 files changed, 141 insertions(+), 71 deletions(-) create mode 100755 python/tests/iotests.sh create mode 100755 tests/qemu-iotests/linters.py -- 2.31.1