Hello Daniel Becker, Jason Fehr, Joe McDonnell, Csaba Ringhofer, Wenzhe Zhou,
Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22358
to look at the new patch set (#11).
Change subject: IMPALA-13672: Migrate query_test/test_kudu.py to use hs2
protocol
......................................................................
IMPALA-13672: Migrate query_test/test_kudu.py to use hs2 protocol
This patch migrate query_test/test_kudu.py to use hs2 client protocol.
Here are the steps taken:
- Override default_test_protocol() to return 'hs2'.
See documentation in ImpalaTestSuite about what this method does.
- Remove usage of deprecated cursor and unique_cursor fixture.
- Replace all direct ImpalaTestSuite.client usage with helper
function call such as execute_query() or execute_query_using_vector().
- Remove all "SET" query invocation and replace it with passing
exec_option dictionary to helper method.
- Replace veryfing kudu modified / inserted rows from reading query
output to reading runtime profile counters.
- Add HS2_TYPES section at test cases where only TYPES exist.
- Remove all drop_impala_table_after_context() calls and replace it with
proper use of unique_database fixture.
KuduTestSuite is fixed with hs2 protocol dimension. Meanwhile,
CustomKuduTest is fixed to use beeswax protocol dimension until proper
migration can be done.
Added following convenience methods:
- ImpalaTestSuite.default_test_protocol() to allow individual test
class to override its default test procol.
- ImpylaHS2ResultSet.tuples() to access the raw HS2 result set that is
a list of tuples.
This patch also added several literal constants around test vector
dimension to help with traceability.
Fixed a bug where "SHOW PARTITIONS" via hs2 over kudu table will shows
NULL number of #Replicas because TResultRowBuilder does not have
overload for int type value. Adjust numFiles variable inside
HdfsTable.getTableStats() from int to long to match Type.BIGINT of
column '#Files'.
Fixed py.test classes that does not inherit BaseTestSuite. Fixed flake8
issues in test_statestore.py.
Testing:
- Run and pass all tests extended from KuduTestSuite in exhaustive mode.
Change-Id: I5f38baf5a0bbde1a1ad0bb4666c300f4f3cabd33
---
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/util/TResultRowBuilder.java
M testdata/workloads/functional-query/queries/QueryTest/kudu-scan-node.test
M
testdata/workloads/functional-query/queries/QueryTest/kudu_create_table_like_table.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_delete.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_update.test
M testdata/workloads/functional-query/queries/QueryTest/kudu_upsert.test
M tests/common/base_test_suite.py
M tests/common/impala_connection.py
M tests/common/impala_test_suite.py
M tests/common/kudu_test_suite.py
M tests/common/test_dimensions.py
M tests/common/test_result_verifier.py
M tests/common/test_vector.py
M tests/conftest.py
M tests/custom_cluster/test_kudu.py
M tests/hs2/test_fetch.py
M tests/infra/test_thrift_parser.py
M tests/infra/test_utils.py
M tests/query_test/test_kudu.py
M tests/query_test/test_observability.py
M tests/statestore/test_statestore.py
M tests/unittests/test_command.py
M tests/verifiers/test_banned_log_messages.py
27 files changed, 1,268 insertions(+), 912 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/22358/11
--
To view, visit http://gerrit.cloudera.org:8080/22358
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5f38baf5a0bbde1a1ad0bb4666c300f4f3cabd33
Gerrit-Change-Number: 22358
Gerrit-PatchSet: 11
Gerrit-Owner: Riza Suminto <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Daniel Becker <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Jason Fehr <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
Gerrit-Reviewer: Wenzhe Zhou <[email protected]>
