Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23842
Change subject: IMPALA-14668: Upgrade to pytest 6.2.5 ...................................................................... IMPALA-14668: Upgrade to pytest 6.2.5 This patch upgrade pytest version from 2.9.2 to 6.2.5, the highest pytest version available without the need to upgrade setuptools. The pytest requirement is moved to py3-requirements.txt and py2-requirements.txt accordingly. However, we can not go back to test with Python2 + pytest-2.9.2 after this patch because our test script need to be adjusted as well. This is OK since we have default testing using Python3 since IMPALA-14333 and stop building impala-shell since IMPALA-14606.The adjustment are follows: - Replace deprecated @pytest.yield_fixture with plain @pytest.fixture. - Replace --resultlog parameter (removed in version 6.0) with --report-log from pytest-reportlog plugin. - Make impala-shell.sh bootstrap Python3 venv (infra/python/env-gcc10.4.0-py3/) by default. Python2 venv (infra/python/env-gcc10.4.0/) is not bootstrapped automatically anymore. - Upgrade execnet to version 1.9.0. This is required by pytest-xdist==2.4.0. - Add -Wonce argument in pytest.ini to slightly suppress warnings. - Add "junit_logging = system-err" option at pytest.ini to continue logging stderr output to junit xml file. With pytest-xdist==2.4.0, parallel EE tests does not show verbose individual test names. It only shows pytest progress marker like following line ...ss....s................................s............................. This is a known limitation in pytest-xdist because execnet, the underlying library used for communication between master and workers, does not support transferring stdout/stderr from workers. https://pytest-xdist.readthedocs.io/en/stable/known-limitations.html Read following links for more detail about the deprecation notes: https://docs.pytest.org/en/stable/deprecations.html This patch also fix small warnings found by pytest-6.2.5 at test_shell_interactive.py and hdfs_util.py. Most of the warnings are about not closing resources properly at the end of test. These warnings should be addressed in follow up patches. Testing: - Pass core BE, FE, and EE tests. Change-Id: Ic3812fe976ef09ac48753dee30151714f4752c24 --- M bin/impala-shell.sh M bin/run-all-tests.sh M infra/python/deps/py2-requirements.txt M infra/python/deps/py3-requirements.txt M infra/python/deps/requirements.txt M tests/comparison/tests/conftest.py M tests/comparison/tests/test_cursor.py M tests/comparison/tests/test_query_objects.py M tests/conftest.py M tests/pytest.ini M tests/query_test/test_observability.py M tests/run-custom-cluster-tests.sh M tests/run-process-failure-tests.sh M tests/run-tests.py M tests/shell/test_shell_commandline.py M tests/shell/test_shell_interactive.py M tests/shell/util.py M tests/util/hdfs_util.py 18 files changed, 73 insertions(+), 49 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/23842/1 -- To view, visit http://gerrit.cloudera.org:8080/23842 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic3812fe976ef09ac48753dee30151714f4752c24 Gerrit-Change-Number: 23842 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto <[email protected]>
