[
https://issues.apache.org/jira/browse/IMPALA-13507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17895557#comment-17895557
]
ASF subversion and git services commented on IMPALA-13507:
----------------------------------------------------------
Commit 95f353ac4a8dd4015368349ef5c138c714891920 in impala's branch
refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=95f353ac4 ]
IMPALA-13507: Allow disabling glog buffering via with_args fixture
We have plenty of custom_cluster tests that assert against content of
Impala daemon log files while the process is still running using
assert_log_contains() and it's wrappers. The method specifically mention
about disabling glog buffering ('-logbuflevel=-1'), but not all
custom_cluster tests do that. This often result in flaky test that hard
to triage and often neglected if it does not frequently run in core
exploration.
This patch adds boolean param 'disable_log_buffering' into
CustomClusterTestSuite.with_args for test to declare intention to
inspect log files in live minicluster. If it is True, start minicluster
with '-logbuflevel=-1' for all daemons. If it is False, log WARNING on
any calls to assert_log_contains().
There are several complex custom_cluster tests that left unchanged and
print out such WARNING logs, such as:
- TestQueryLive
- TestQueryLogTableBeeswax
- TestQueryLogOtherTable
- TestQueryLogTableHS2
- TestQueryLogTableAll
- TestQueryLogTableBufferPool
- TestStatestoreRpcErrors
- TestWorkloadManagementInitWait
- TestWorkloadManagementSQLDetails
This patch also fixed some small flake8 issues on modified tests.
There is a flakiness sign at test_query_live.py where test query is
submitted to coordinator and fail because sys.impala_query_live table
has not exist yet from coordinator's perspective. This patch modify
test_query_live.py to wait for few seconds until sys.impala_query_live
is queryable.
Testing:
- Pass custom_cluster tests in exhaustive exploration.
Change-Id: I56fb1746b8f3cea9f3db3514a86a526dffb44a61
Reviewed-on: http://gerrit.cloudera.org:8080/22015
Reviewed-by: Jason Fehr <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add param to disable glog buffering in with_args fixture
> --------------------------------------------------------
>
> Key: IMPALA-13507
> URL: https://issues.apache.org/jira/browse/IMPALA-13507
> Project: IMPALA
> Issue Type: Improvement
> Components: Test
> Affects Versions: Impala 4.4.0
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
>
> We have plenty of custom_cluster tests that assert against content of Impala
> daemon log files while the process is still running using
> assert_log_contains() and it's wrappers. The method specifically mention
> about disabling glog buffering ('-logbuflevel=-1'), but not all
> custom_cluster tests do that.
> {code:java}
> def assert_log_contains(self, daemon, level, line_regex, expected_count=1,
> timeout_s=6,
> dry_run=False):
> """
> Assert that the daemon log with specified level (e.g. ERROR, WARNING,
> INFO) contains
> expected_count lines with a substring matching the regex. When
> expected_count is -1,
> at least one match is expected.
> Retries until 'timeout_s' has expired. The default timeout is the default
> minicluster
> log buffering time (5 seconds) with a one second buffer.
> When using this method to check log files of running processes, the
> caller should
> make sure that log buffering has been disabled, for example by adding
> '-logbuflevel=-1' to the daemon startup options or set timeout_s to a
> value higher
> than the log flush interval. Returns the result of the very last call
> to line_regex.search or None if
> expected_count is 0 or the line_regex did not match any lines.
> """ {code}
> This often result in flaky test that hard to triage and often neglected if it
> does not frequently run in core exploration.
> We can improve this by adding boolean param into
> CustomClusterTestSuite.with_args, say 'disable_log_buffering', for test to
> declare intention to inspect log files in live minicluster. If it is True,
> start minicluster with '-logbuflevel=-1' for all daemons. If it is False, log
> WARNING on any calls to assert_log_contains().
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
