Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19567 )
Change subject: IMPALA-11959: Add Python 3 virtualenv ...................................................................... IMPALA-11959: Add Python 3 virtualenv This adds a Python 3 equivalent to the impala-python virtualenv base on the toolchain Python 3.7.16. This modifies bootstrap_virtualenv.py to support the two different modes. This adds py2-requirements.txt and py3-requirements.txt to allow some differences between the Python 2 and Python 3 virtualenvs. Here are some specific package changes: - allpairs is replaced with allpairspy, as allpairs did not support Python 3. - requests is upgraded slightly, because otherwise is has issues with idna==2.8. - pylint is limited to Python 3, because we are adding it and don't need it on both - flake8 is limited to Python 2, because it will take some work to switch to a version that works on Python 3 - cm_api is limited to Python 2, because it doesn't support Python 3 - pytest-random does not support Python 3 and it is unused, so it is removed - Bump the version of setuptool-scm to support Python 3 This adds impala-pylint, which can be used to do further Python 3 checks via --py3k. This also adds a bin/check-pylint-py3k.sh script to enforce specific py3k checks. The banned py3k warnings are specified in the bin/banned_py3k_warnings.txt. This is currently empty, but this can ratchet up the py3k strictness over time to avoid regressions. This pulls in a new toolchain with the fix for IMPALA-11956 to get Python 3.7.16. Testing: - Hand tested that the allpairs libraries produce the same results - The python3 virtualenv has no influence on regular tests yet Change-Id: Ica4853f440c9a46a79bd5fb8e0a66730b0b4efc0 Reviewed-on: http://gerrit.cloudera.org:8080/19567 Reviewed-by: Joe McDonnell <[email protected]> Tested-by: Joe McDonnell <[email protected]> --- M CMakeLists.txt A bin/banned_py3k_warnings.txt M bin/bootstrap_toolchain.py A bin/check-pylint-py3k.sh M bin/impala-config.sh A bin/impala-pip3 A bin/impala-pylint A bin/impala-python3 A bin/impala-python3-common.sh M bin/init-impala-python.sh M bin/rat_exclude_files.txt M infra/python/bootstrap_virtualenv.py M infra/python/deps/pip_download.py A infra/python/deps/py2-requirements.txt A infra/python/deps/py3-requirements.txt M infra/python/deps/requirements.txt M infra/python/deps/setuptools-requirements.txt M testdata/bin/generate-test-vectors.py M tests/common/test_vector.py M tests/custom_cluster/test_hs2_fault_injection.py M tests/query_test/test_decimal_casting.py 21 files changed, 452 insertions(+), 75 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/19567 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ica4853f440c9a46a79bd5fb8e0a66730b0b4efc0 Gerrit-Change-Number: 19567 Gerrit-PatchSet: 7 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Laszlo Gaal <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]>
