Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21474 )
Change subject: IMPALA-13123: Add option to run tests with Python 3 ...................................................................... IMPALA-13123: Add option to run tests with Python 3 This introduces the IMPALA_USE_PYTHON3_TESTS environment variable to select whether to run tests using the toolchain Python 3. This is an experimental option, so it defaults to false, continuing to run tests with Python 2. This fixes a first batch of Python 2 vs 3 issues: - Deciding whether to open a file in bytes mode or text mode - Adapting to APIs that operate on bytes in Python 3 (e.g. codecs) - Eliminating 'basestring' and 'unicode' locations in tests/ by using the recommendations from future ( https://python-future.org/compatible_idioms.html#basestring and https://python-future.org/compatible_idioms.html#unicode ) - Uses impala-python3 for bin/start-impala-cluster.py All fixes leave the Python 2 path working normally. Testing: - Ran an exhaustive run with Python 2 to verify nothing broke - Verified that the new environment variable works and that it uses Python 3 from the toolchain when specified Change-Id: I177d9b8eae9b99ba536ca5c598b07208c3887f8c Reviewed-on: http://gerrit.cloudera.org:8080/21474 Reviewed-by: Michael Smith <[email protected]> Reviewed-by: Riza Suminto <[email protected]> Tested-by: Joe McDonnell <[email protected]> --- M bin/impala-config.sh A bin/impala-env-versioned-python M bin/impala-py.test M bin/start-impala-cluster.py M lib/python/impala_py_lib/helpers.py M testdata/bin/generate-schema-statements.py M tests/beeswax/impala_beeswax.py M tests/common/impala_connection.py M tests/common/impala_test_suite.py M tests/common/resource_pool_config.py M tests/common/test_dimensions.py M tests/common/test_result_verifier.py M tests/conftest.py M tests/custom_cluster/test_parquet_max_page_header.py M tests/custom_cluster/test_query_log.py M tests/data_errors/test_data_errors.py M tests/hs2/hs2_test_suite.py M tests/hs2/test_hs2.py M tests/query_test/test_insert_parquet.py M tests/query_test/test_observability.py M tests/query_test/test_parquet_bloom_filter.py M tests/query_test/test_parquet_page_index.py M tests/query_test/test_scanners.py M tests/run-tests.py M tests/shell/util.py M tests/util/get_parquet_metadata.py M tests/util/hdfs_util.py M tests/util/shell_util.py M tests/util/test_file_parser.py 29 files changed, 167 insertions(+), 82 deletions(-) Approvals: Michael Smith: Looks good to me, but someone else must approve Riza Suminto: Looks good to me, approved Joe McDonnell: Verified -- To view, visit http://gerrit.cloudera.org:8080/21474 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I177d9b8eae9b99ba536ca5c598b07208c3887f8c Gerrit-Change-Number: 21474 Gerrit-PatchSet: 13 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]>
