Hello Jason Fehr, Joe McDonnell, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23842
to look at the new patch set (#2).
Change subject: IMPALA-14668: Upgrade to pytest 6.2.5
......................................................................
IMPALA-14668: Upgrade to pytest 6.2.5
This patch upgrade pytest version from 2.9.2 to 6.2.5, the highest
pytest version available without the need to upgrade setuptools.
The pytest requirement is moved to py3-requirements.txt and
py2-requirements.txt accordingly. However, we can not go back to test
with Python2 + pytest-2.9.2 after this patch because our test script
need to be adjusted as well. This is OK since we have default testing
using Python3 since IMPALA-14333 and stop building impala-shell since
IMPALA-14606. The adjustment are follows:
- Replace deprecated @pytest.yield_fixture with plain @pytest.fixture.
- Replace --resultlog parameter (removed in version 6.0) with
--report-log from pytest-reportlog plugin.
- Make impala-shell.sh bootstrap Python3
venv (infra/python/env-gcc10.4.0-py3/) by default. Python2
venv (infra/python/env-gcc10.4.0/) is not bootstrapped automatically
anymore.
- Upgrade execnet to version 1.9.0. This is required by
pytest-xdist==2.4.0.
- Add -Wonce argument in pytest.ini to slightly suppress warnings.
- Add "junit_logging = system-err" option at pytest.ini to continue
logging stderr output to junit xml file.
With pytest-xdist==2.4.0, parallel EE tests does not show verbose
individual test names. It only shows pytest progress marker like
following line
...ss....s................................s.............................
This is a known limitation in pytest-xdist because execnet, the
underlying library used for communication between master and workers,
does not support transferring stdout/stderr from workers.
https://pytest-xdist.readthedocs.io/en/stable/known-limitations.html
Read following links for more detail about the deprecation notes:
https://docs.pytest.org/en/stable/deprecations.html
Change SKIP_SSL_MSG default to empty string because skipif does not
accept None reason anymore. Fixed erroneous log formatting in
test_restart_services.py.
This patch also fix small warnings found by pytest-6.2.5 at
test_shell_interactive.py and hdfs_util.py. Most of the warnings are
about not closing resources properly at the end of test. These warnings
should be addressed in follow up patches.
Testing:
- Pass core BE, FE, and EE tests.
Change-Id: Ic3812fe976ef09ac48753dee30151714f4752c24
---
M bin/impala-shell.sh
M bin/run-all-tests.sh
M infra/python/deps/py2-requirements.txt
M infra/python/deps/py3-requirements.txt
M infra/python/deps/requirements.txt
M tests/common/network.py
M tests/comparison/tests/conftest.py
M tests/comparison/tests/test_cursor.py
M tests/comparison/tests/test_query_objects.py
M tests/conftest.py
M tests/custom_cluster/test_client_ssl.py
M tests/custom_cluster/test_ipv6.py
M tests/custom_cluster/test_restart_services.py
M tests/pytest.ini
M tests/query_test/test_observability.py
M tests/run-custom-cluster-tests.sh
M tests/run-process-failure-tests.sh
M tests/run-tests.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/util/hdfs_util.py
22 files changed, 87 insertions(+), 61 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/23842/2
--
To view, visit http://gerrit.cloudera.org:8080/23842
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic3812fe976ef09ac48753dee30151714f4752c24
Gerrit-Change-Number: 23842
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Jason Fehr <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
