Hello Jason Fehr, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23319
to look at the new patch set (#3).
Change subject: IMPALA-14333: [part 1] Make all pytest compatible with Python3
......................................................................
IMPALA-14333: [part 1] Make all pytest compatible with Python3
Running exhaustive tests with env var IMPALA_USE_PYTHON3_TESTS=true
reveals some tests that require adjustment. This patch made such
adjustment, which mostly revolves around encoding difference and string
vs bytes type in Python3. Followings are the details:
Moved one test case from delimited-latin-text.test to
test_delimited_text.py for easier binary comparison.
Set timezone='Europe/Budapest' in test_sort and test_top_n to get
consistent result for reading functional_parquet.complextypes_structs
and functional_orc_def.complextypes_structs.
Skip testing HBase at test_except and test_partitioned_top_n because
HBase does not return return rows where alltypesagg.tinyint_col is null.
Add _run_query_with_client() in test_ranger.py to allow reusing single
Impala client for running several queries. Make sure to close the
clients when test is done. Mark several tests in test_ranger.py with
SkipIfFS.hive because they run query through beeline + HiveServer2, but
Ozone and S3 build environment does not start HiveServer2 by default.
Add bytes_to_str() as utility function to decode bytes in Python3. This
often needed when inspecting return value of subprocess.check_output()
as string.
Implement DataTypeMetaclass.__lt__ to substitute
DataTypeMetaclass.__cmp__ that is ignored in Python3 (see
https://peps.python.org/pep-0207/).
Fix WEB_CERT_ERR difference in test_ipv6.py.
Fix trivial integer parsing in test_restart_services.py.
Fix various encoding issues in test_saml2_sso.py,
test_shell_commandline.py, and test_shell_interactive.py.
Switch parallel loops in test_stress_infra.py to simple loop that
fail-proof in Python3.
Reduce test dimension in test_date_queries.py and
test_decimal_queries.py for json file format because table date_tbl and
decimal_tiny is contrained only for json/none/none (functional_json).
Switch to binary comparison in test_iceberg.py where needed.
Specify text mode when calling tempfile.NamedTemporaryFile().
Increase sleep period from 0.1 to 0.5 second per iteration in
test_statestore.py because TServer seem to shutdown slower. Remove
deprecated setDaemon() calls.
Removed unused vector argument and fix some trivial flake8 issues.
Simplify create_impala_shell_executable_dimension to skip testing
python2 impala-shell when IMPALA_USE_PYTHON3_TESTS=true.
Switching IMPALA_USE_PYTHON3_TESTS to true by default will be addressed
in next patch.
Testing:
Pass exhaustive tests with IMPALA_USE_PYTHON3_TESTS=true.
Change-Id: I401a93b6cc7bcd17f41d24e7a310e0c882a550d4
---
M shell/impala_shell/impala_shell.py
M
testdata/workloads/functional-query/queries/QueryTest/delimited-latin-text.test
M testdata/workloads/functional-query/queries/QueryTest/inline-view.test
M testdata/workloads/functional-query/queries/QueryTest/sort-complex.test
M testdata/workloads/functional-query/queries/QueryTest/top-n-complex.test
M tests/authorization/test_ranger.py
M tests/common/impala_test_suite.py
M tests/common/skip.py
M tests/comparison/cluster.py
M tests/comparison/db_types.py
M tests/custom_cluster/test_ipv6.py
M tests/custom_cluster/test_restart_services.py
M tests/custom_cluster/test_s3a_access.py
M tests/custom_cluster/test_saml2_sso.py
M tests/custom_cluster/test_scratch_disk.py
M tests/infra/test_stress_infra.py
M tests/query_test/test_date_queries.py
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_delimited_text.py
M tests/query_test/test_iceberg.py
M tests/query_test/test_queries.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/statestore/test_statestore.py
M tests/stress/test_update_stress.py
M tests/util/hdfs_util.py
M tests/util/parse_util.py
M tests/util/shell_util.py
29 files changed, 364 insertions(+), 243 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/23319/3
--
To view, visit http://gerrit.cloudera.org:8080/23319
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I401a93b6cc7bcd17f41d24e7a310e0c882a550d4
Gerrit-Change-Number: 23319
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Jason Fehr <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
