Title: [267493] trunk/Tools
- Revision
- 267493
- Author
- aakash_j...@apple.com
- Date
- 2020-09-23 12:17:20 -0700 (Wed, 23 Sep 2020)
Log Message
Limit number of emails to send for flaky and pre-existing API and layout test failures
https://bugs.webkit.org/show_bug.cgi?id=216876
Reviewed by Jonathan Bedard.
* BuildSlaveSupport/ews-build/steps.py:
(ReRunWebKitTests): Simply limit the flaky failures immediately after they are calculated, since we don't display
the count of flaky failures. Flaky failures are displayed only to help bot-watchers in noticing flaky tests.
(AnalyzeLayoutTestsResults.report_pre_existing_failures): Limit pre-existing and flaky test failures just before sending emails.
(AnalyzeAPITestsResults.analyzeResults):
Modified Paths
Diff
Modified: trunk/Tools/BuildSlaveSupport/ews-build/steps.py (267492 => 267493)
--- trunk/Tools/BuildSlaveSupport/ews-build/steps.py 2020-09-23 19:07:10 UTC (rev 267492)
+++ trunk/Tools/BuildSlaveSupport/ews-build/steps.py 2020-09-23 19:17:20 UTC (rev 267493)
@@ -2041,6 +2041,7 @@
class ReRunWebKitTests(RunWebKitTests):
name = 're-run-layout-tests'
+ NUM_FAILURES_TO_DISPLAY = 10
def evaluateCommand(self, cmd):
rc = self.evaluateResult(cmd)
@@ -2050,6 +2051,7 @@
second_results_failing_tests = set(self.getProperty('second_run_failures', []))
tests_that_consistently_failed = first_results_failing_tests.intersection(second_results_failing_tests)
flaky_failures = first_results_failing_tests.union(second_results_failing_tests) - first_results_failing_tests.intersection(second_results_failing_tests)
+ flaky_failures = list(flaky_failures)[:self.NUM_FAILURES_TO_DISPLAY]
flaky_failures_string = ', '.join(flaky_failures)
if rc == SUCCESS or rc == WARNINGS:
@@ -2164,7 +2166,7 @@
message = 'Found {} pre-existing test failure{}: {}'.format(len(clean_tree_failures), pluralSuffix, clean_tree_failures_string)
if len(clean_tree_failures) > self.NUM_FAILURES_TO_DISPLAY:
message += ' ...'
- for clean_tree_failure in clean_tree_failures:
+ for clean_tree_failure in list(clean_tree_failures)[:self.NUM_FAILURES_TO_DISPLAY]:
self.send_email_for_pre_existing_failure(clean_tree_failure)
if flaky_failures:
@@ -2173,7 +2175,7 @@
message += ' Found flaky test{}: {}'.format(pluralSuffix, flaky_failures_string)
if len(flaky_failures) > self.NUM_FAILURES_TO_DISPLAY:
message += ' ...'
- for flaky_failure in flaky_failures:
+ for flaky_failure in list(flaky_failures)[:self.NUM_FAILURES_TO_DISPLAY]:
self.send_email_for_flaky_failure(flaky_failure)
self.setProperty('build_summary', message)
@@ -2589,6 +2591,7 @@
failures_with_patch = first_run_failures.intersection(second_run_failures)
flaky_failures = first_run_failures.union(second_run_failures) - first_run_failures.intersection(second_run_failures)
+ flaky_failures = list(flaky_failures)[:self.NUM_API_FAILURES_TO_DISPLAY]
flaky_failures_string = ', '.join(flaky_failures)
new_failures = failures_with_patch - clean_tree_failures
new_failures_to_display = list(new_failures)[:self.NUM_API_FAILURES_TO_DISPLAY]
@@ -2618,7 +2621,7 @@
message = ''
if clean_tree_failures:
message = 'Found {} pre-existing API test failure{}: {}'.format(len(clean_tree_failures), pluralSuffix, clean_tree_failures_string)
- for clean_tree_failure in clean_tree_failures:
+ for clean_tree_failure in clean_tree_failures_to_display:
self.send_email_for_pre_existing_failure(clean_tree_failure)
if len(clean_tree_failures) > self.NUM_API_FAILURES_TO_DISPLAY:
message += ' ...'
Modified: trunk/Tools/ChangeLog (267492 => 267493)
--- trunk/Tools/ChangeLog 2020-09-23 19:07:10 UTC (rev 267492)
+++ trunk/Tools/ChangeLog 2020-09-23 19:17:20 UTC (rev 267493)
@@ -1,5 +1,18 @@
2020-09-23 Aakash Jain <aakash_j...@apple.com>
+ Limit number of emails to send for flaky and pre-existing API and layout test failures
+ https://bugs.webkit.org/show_bug.cgi?id=216876
+
+ Reviewed by Jonathan Bedard.
+
+ * BuildSlaveSupport/ews-build/steps.py:
+ (ReRunWebKitTests): Simply limit the flaky failures immediately after they are calculated, since we don't display
+ the count of flaky failures. Flaky failures are displayed only to help bot-watchers in noticing flaky tests.
+ (AnalyzeLayoutTestsResults.report_pre_existing_failures): Limit pre-existing and flaky test failures just before sending emails.
+ (AnalyzeAPITestsResults.analyzeResults):
+
+2020-09-23 Aakash Jain <aakash_j...@apple.com>
+
Notify igalia team about pre-existing test failures and build failures for JSC queues
https://bugs.webkit.org/show_bug.cgi?id=216889
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
