Hello Daniel Becker, Joe McDonnell, Csaba Ringhofer, Wenzhe Zhou, Impala Public
Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22358
to look at the new patch set (#7).
Change subject: IMPALA-13672: Migrate query_test/test_kudu.py to use hs2
protocol
......................................................................
IMPALA-13672: Migrate query_test/test_kudu.py to use hs2 protocol
This patch migrate query_test/test_kudu.py to use hs2 client protocol.
Here are the steps taken:
- Override default_test_protocol() to return 'hs2'.
See documentation in ImpalaTestSuite about what this method does.
- Remove usage of deprecated cursor and unique_cursor fixture.
- Replace all direct ImpalaTestSuite.cient usage with helper
function call such as execute_query() or execute_query_using_vector().
- Remove all "SET" query invocation and replace it with passing
exec_option dictionary to helper method.
- Replace veryfing kudu modified / inserted rows from reading query
output to reading runtime profile counters.
- Add HS2_TYPES section at test cases where only TYPES exist.
- Remove all drop_impala_table_after_context() calls and replace it with
proper use of unique_database fixture.
KuduTestSuite is fixed with hs2 protocol dimension. Meanwhile,
CustomKuduTest is fixed to use beeswax protocol dimension until proper
migration can be done.
This patch added several new methods in ImpalaTestSuite:
- default_test_protocol() to allow individual test class to override its
default test procol.
- execute_query_using_vector() for querying using default client that
match 'protocol' dimension while configuring it with 'exec_option'
dimension.
- create_impala_client_from_vector() for creating new client based on
'protocol' dimension of vector.
This patch also added several literal constants around test vector
dimension to help with traceability.
Catched a bug where "SHOW PARTITIONS" via hs2 over kudu table will shows
NULL number of #Replicas because TResultRowBuilder does not have
overload for int type value. Adjust numFiles variable inside
HdfsTable.getTableStats() from int to long to match Type.BIGINT of
column '#Files'.
Fixed py.test classes that does not inherit BaseTestSuite. Fixed flake8
issues in test_statestore.py.
Testing:
- Run and pass all tests extended from KuduTestSuite in exhaustive mode.
Change-Id: I5f38baf5a0bbde1a1ad0bb4666c300f4f3cabd33
---
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/util/TResultRowBuilder.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M
testdata/workloads/functional-query/queries/QueryTest/kudu_create_table_like_table.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/common/base_test_suite.py
M tests/common/impala_connection.py
M tests/common/impala_test_suite.py
M tests/common/kudu_test_suite.py
M tests/common/test_dimensions.py
M tests/common/test_result_verifier.py
M tests/common/test_vector.py
M tests/conftest.py
M tests/custom_cluster/test_kudu.py
M tests/hs2/test_fetch.py
M tests/infra/test_thrift_parser.py
M tests/infra/test_utils.py
M tests/query_test/test_kudu.py
M tests/query_test/test_observability.py
M tests/statestore/test_statestore.py
M tests/unittests/test_command.py
M tests/verifiers/test_banned_log_messages.py
27 files changed, 1,283 insertions(+), 917 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/22358/7
--
To view, visit http://gerrit.cloudera.org:8080/22358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5f38baf5a0bbde1a1ad0bb4666c300f4f3cabd33
Gerrit-Change-Number: 22358
Gerrit-PatchSet: 7
Gerrit-Owner: Riza Suminto <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Daniel Becker <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
Gerrit-Reviewer: Wenzhe Zhou <[email protected]>
