Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/22015 )
Change subject: IMPALA-13507: Allow disabling glog buffering via with_args
fixture
......................................................................
IMPALA-13507: Allow disabling glog buffering via with_args fixture
We have plenty of custom_cluster tests that assert against content of
Impala daemon log files while the process is still running using
assert_log_contains() and it's wrappers. The method specifically mention
about disabling glog buffering ('-logbuflevel=-1'), but not all
custom_cluster tests do that. This often result in flaky test that hard
to triage and often neglected if it does not frequently run in core
exploration.
This patch adds boolean param 'disable_log_buffering' into
CustomClusterTestSuite.with_args for test to declare intention to
inspect log files in live minicluster. If it is True, start minicluster
with '-logbuflevel=-1' for all daemons. If it is False, log WARNING on
any calls to assert_log_contains().
There are several complex custom_cluster tests that left unchanged and
print out such WARNING logs, such as:
- TestQueryLive
- TestQueryLogTableBeeswax
- TestQueryLogOtherTable
- TestQueryLogTableHS2
- TestQueryLogTableAll
- TestQueryLogTableBufferPool
- TestStatestoreRpcErrors
- TestWorkloadManagementInitWait
- TestWorkloadManagementSQLDetails
This patch also fixed some small flake8 issues on modified tests.
There is a flakiness sign at test_query_live.py where test query is
submitted to coordinator and fail because sys.impala_query_live table
has not exist yet from coordinator's perspective. This patch modify
test_query_live.py to wait for few seconds until sys.impala_query_live
is queryable.
Testing:
- Pass custom_cluster tests in exhaustive exploration.
Change-Id: I56fb1746b8f3cea9f3db3514a86a526dffb44a61
Reviewed-on: http://gerrit.cloudera.org:8080/22015
Reviewed-by: Jason Fehr <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
M tests/authorization/test_authorization.py
M tests/authorization/test_ranger.py
M tests/common/custom_cluster_test_suite.py
M tests/common/impala_test_suite.py
M tests/custom_cluster/test_admission_controller.py
M tests/custom_cluster/test_catalog_hms_failures.py
M tests/custom_cluster/test_catalogd_ha.py
M tests/custom_cluster/test_codegen_cache.py
M tests/custom_cluster/test_custom_statestore.py
M tests/custom_cluster/test_data_cache.py
M tests/custom_cluster/test_disable_catalog_data_ops.py
M tests/custom_cluster/test_events_custom_configs.py
M tests/custom_cluster/test_local_catalog.py
M tests/custom_cluster/test_logging.py
M tests/custom_cluster/test_partition.py
M tests/custom_cluster/test_pause_monitor.py
M tests/custom_cluster/test_query_event_hooks.py
M tests/custom_cluster/test_query_expiration.py
M tests/custom_cluster/test_query_live.py
M tests/custom_cluster/test_query_log.py
M tests/custom_cluster/test_query_retries.py
M tests/custom_cluster/test_re2_max_mem.py
M tests/custom_cluster/test_restart_services.py
M tests/custom_cluster/test_runtime_filter_aggregation.py
M tests/custom_cluster/test_services_rpc_errors.py
M tests/custom_cluster/test_shell_jwt_auth.py
M tests/custom_cluster/test_thrift_socket.py
M tests/custom_cluster/test_workload_mgmt_init.py
28 files changed, 163 insertions(+), 80 deletions(-)
Approvals:
Jason Fehr: Looks good to me, but someone else must approve
Michael Smith: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/22015
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I56fb1746b8f3cea9f3db3514a86a526dffb44a61
Gerrit-Change-Number: 22015
Gerrit-PatchSet: 9
Gerrit-Owner: Riza Suminto <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Jason Fehr <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
