[
https://issues.apache.org/jira/browse/IMPALA-14182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18088298#comment-18088298
]
ASF subversion and git services commented on IMPALA-14182:
----------------------------------------------------------
Commit ec104280d9d072359a04c16b6b37c5b760c30b6a in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ec104280d ]
IMPALA-14182: Deflake test_breakpad.py's TestLogging* tests
These tests have been flaky with two types of error:
- Exceeding the max expected log files
- Exceeding the max expected log file size
This relaxes those two checks:
- The enforcement of the max log files is done by a background
thread, and it can briefly exceed the number of log files.
This should be limited to a single log file per severity
level. This adjusts the max log file calculation to allow
for an extra log file per severity level.
- This increases the max log file size from 1.2MB to 1.3MB.
This also changes the tests to preserve the logs if hitting
any of these asserts. This should make it easier to tell
what is happening if this reoccurs.
Testing:
- Ran TestLogging/TestLoggingExhaustive locally
- Forced a failure and verified that the logs are preserved
Change-Id: I79f97ca26320369ebb53f7f57e6c86e912c9cf02
Reviewed-on: http://gerrit.cloudera.org:8080/24437
Reviewed-by: Michael Smith <[email protected]>
Reviewed-by: Csaba Ringhofer <[email protected]>
Tested-by: Joe McDonnell <[email protected]>
> custom_cluster.test_breakpad.TestLogging.test_excessive_cerr_ignore_pid and
> test_excessive_cerr_no_symlink seem to be flaky
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-14182
> URL: https://issues.apache.org/jira/browse/IMPALA-14182
> Project: IMPALA
> Issue Type: Bug
> Reporter: Fang-Yu Rao
> Assignee: Joe McDonnell
> Priority: Major
> Labels: broken-build
>
> We found that {{test_excessive_cerr_ignore_pid()}} in
> [https://github.com/apache/impala/blob/master/tests/custom_cluster/test_breakpad.py]
> could fail with the following error message and stack trace. It looks like
> we could have more log files (5) than expected (4).
> +*Error Message*+
> {code:java}
> assert 5 <= 4 + where 5 =
> len(['/data/jenkins/workspace/impala-cdw-master-core-ubsan-arm/repos/Impala/logs/custom_cluster_tests/impala_test_breakpad..._1mhwju/impalad.impala-ec2-rhel88-m7g-4xlarge-ondemand-012c.vpc.cloudera.com.jenkins.log.INFO.20250623-152709.3603455'])
> {code}
> +*Stacktrace*+
> {code:java}
> custom_cluster/test_breakpad.py:521: in test_excessive_cerr_ignore_pid
> match_pid=False)
> custom_cluster/test_breakpad.py:499: in start_excessive_cerr_cluster
> self.assert_logs(daemon, max_log_count_end, expected_log_max_bytes)
> custom_cluster/test_breakpad.py:409: in assert_logs
> assert len(log_paths) <= max_count
> E assert 5 <= 4
> E + where 5 =
> len(['/data/jenkins/workspace/impala-cdw-master-core-ubsan-arm/repos/Impala/logs/custom_cluster_tests/impala_test_breakpad..._1mhwju/impalad.impala-ec2-rhel88-m7g-4xlarge-ondemand-012c.vpc.cloudera.com.jenkins.log.INFO.20250623-152709.3603455'])
> {code}
>
> {{test_excessive_cerr_no_symlink()}} could also fail with a similar symptom.
> +*Error Message*+
> {code:java}
> AssertionError:
> /data/jenkins/workspace/impala-cdw-master-staging-exhaustive-release-arm/repos/Impala/logs/custom_cluster_tests/impala_test_breakpad_Zxp6uT/impalad.impala-ec2-rhel88-m7g-4xlarge-ondemand-1196.vpc.cloudera.com.jenkins.log.ERROR.20250625-081903.2696592
> exceed 1258291 bytes assert 1276951 <= 1258291
> {code}
> +*Stacktrace*+
> {code:java}
> custom_cluster/test_breakpad.py:554: in test_excessive_cerr_no_symlink
> self.start_excessive_cerr_cluster(test_cluster_size=2,
> remove_symlink=False)
> custom_cluster/test_breakpad.py:499: in start_excessive_cerr_cluster
> self.assert_logs(daemon, max_log_count_end, expected_log_max_bytes)
> custom_cluster/test_breakpad.py:427: in assert_logs
> assert log_size <= max_bytes, "{} exceed {} bytes".format(curr_path,
> max_bytes)
> E AssertionError:
> /data/jenkins/workspace/impala-cdw-master-staging-exhaustive-release-arm/repos/Impala/logs/custom_cluster_tests/impala_test_breakpad_Zxp6uT/impalad.impala-ec2-rhel88-m7g-4xlarge-ondemand-1196.vpc.cloudera.com.jenkins.log.ERROR.20250625-081903.2696592
> exceed 1258291 bytes
> E assert 1276951 <= 1258291
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
