Title: [244306] trunk/Tools
- Revision
- 244306
- Author
- [email protected]
- Date
- 2019-04-15 16:36:25 -0700 (Mon, 15 Apr 2019)
Log Message
ews-build should clearly indicate flaky test failures
https://bugs.webkit.org/show_bug.cgi?id=196947
Reviewed by Lucas Forschler.
* BuildSlaveSupport/ews-build/steps.py:
(AnalyzeAPITestsResults.analyzeResults):
Modified Paths
Diff
Modified: trunk/Tools/BuildSlaveSupport/ews-build/steps.py (244305 => 244306)
--- trunk/Tools/BuildSlaveSupport/ews-build/steps.py 2019-04-15 23:15:21 UTC (rev 244305)
+++ trunk/Tools/BuildSlaveSupport/ews-build/steps.py 2019-04-15 23:36:25 UTC (rev 244306)
@@ -792,13 +792,17 @@
second_run_failures = getAPITestFailures(second_run_results)
clean_tree_failures = getAPITestFailures(clean_tree_results)
- self._addToLog('stderr', '\nFailures in API Test first run: {}'.format(first_run_failures))
- self._addToLog('stderr', '\nFailures in API Test second run: {}'.format(first_run_failures))
- self._addToLog('stderr', '\nFailures in API Test on clean tree: {}'.format(clean_tree_failures))
failures_with_patch = first_run_failures.intersection(second_run_failures)
+ flaky_failures = first_run_failures.union(second_run_failures) - first_run_failures.intersection(second_run_failures)
+ flaky_failures_string = ', '.join([failure_name.replace('TestWebKitAPI.', '') for failure_name in flaky_failures])
new_failures = failures_with_patch - clean_tree_failures
new_failures_string = ', '.join([failure_name.replace('TestWebKitAPI.', '') for failure_name in new_failures])
+ self._addToLog('stderr', '\nFailures in API Test first run: {}'.format(first_run_failures))
+ self._addToLog('stderr', '\nFailures in API Test second run: {}'.format(second_run_failures))
+ self._addToLog('stderr', '\nFlaky Tests: {}'.format(flaky_failures))
+ self._addToLog('stderr', '\nFailures in API Test on clean tree: {}'.format(clean_tree_failures))
+
if new_failures:
self._addToLog('stderr', '\nNew failures: {}\n'.format(new_failures))
self.finished(FAILURE)
@@ -812,6 +816,8 @@
self.build.results = SUCCESS
self.descriptionDone = 'Passed API tests'
message = 'Found {} pre-existing API tests failures'.format(len(clean_tree_failures))
+ if flaky_failures:
+ message += '. Flaky tests: {}'.format(flaky_failures_string)
self.build.buildFinished([message], SUCCESS)
@defer.inlineCallbacks
Modified: trunk/Tools/ChangeLog (244305 => 244306)
--- trunk/Tools/ChangeLog 2019-04-15 23:15:21 UTC (rev 244305)
+++ trunk/Tools/ChangeLog 2019-04-15 23:36:25 UTC (rev 244306)
@@ -1,3 +1,13 @@
+2019-04-15 Aakash Jain <[email protected]>
+
+ ews-build should clearly indicate flaky test failures
+ https://bugs.webkit.org/show_bug.cgi?id=196947
+
+ Reviewed by Lucas Forschler.
+
+ * BuildSlaveSupport/ews-build/steps.py:
+ (AnalyzeAPITestsResults.analyzeResults):
+
2019-04-15 Brent Fulgham <[email protected]>
InjectedBundle parameters often need initialization function called before unarchiving
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
