Title: [284227] trunk/Tools
- Revision
- 284227
- Author
- [email protected]
- Date
- 2021-10-14 19:20:48 -0700 (Thu, 14 Oct 2021)
Log Message
[webkitpy] Occasional exception thrown in change_result_to_failure when repeating flaky layout tests
https://bugs.webkit.org/show_bug.cgi?id=229788
Reviewed by Jonathan Bedard.
When a iterated/repeated flaky test moves from an unexpected Failure to
successive unexpected Pass, the currently stored result is still the
Failure (likely due to r235467). This may cause change_result_to_failure
to raise KeyError trying to forcibly remove() the Failure result from
the test_by_expectations set twice.
This commit relaxes the behavior to just ask to discard the value from
the set.
* Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py:
(LayoutTestRunnerTests.test_update_summary_with_result): Added test case.
* Scripts/webkitpy/layout_tests/models/test_run_results.py:
(TestRunResults.change_result_to_failure): Use set.discard to avoid
KeyErrors.
Modified Paths
Diff
Modified: trunk/Tools/ChangeLog (284226 => 284227)
--- trunk/Tools/ChangeLog 2021-10-15 02:02:14 UTC (rev 284226)
+++ trunk/Tools/ChangeLog 2021-10-15 02:20:48 UTC (rev 284227)
@@ -1,3 +1,25 @@
+2021-10-14 Lauro Moura <[email protected]>
+
+ [webkitpy] Occasional exception thrown in change_result_to_failure when repeating flaky layout tests
+ https://bugs.webkit.org/show_bug.cgi?id=229788
+
+ Reviewed by Jonathan Bedard.
+
+ When a iterated/repeated flaky test moves from an unexpected Failure to
+ successive unexpected Pass, the currently stored result is still the
+ Failure (likely due to r235467). This may cause change_result_to_failure
+ to raise KeyError trying to forcibly remove() the Failure result from
+ the test_by_expectations set twice.
+
+ This commit relaxes the behavior to just ask to discard the value from
+ the set.
+
+ * Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py:
+ (LayoutTestRunnerTests.test_update_summary_with_result): Added test case.
+ * Scripts/webkitpy/layout_tests/models/test_run_results.py:
+ (TestRunResults.change_result_to_failure): Use set.discard to avoid
+ KeyErrors.
+
2021-10-14 Chris Dumez <[email protected]>
Service workers running on the main thread should use the main VM
Modified: trunk/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py (284226 => 284227)
--- trunk/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py 2021-10-15 02:02:14 UTC (rev 284226)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner_unittest.py 2021-10-15 02:20:48 UTC (rev 284227)
@@ -126,7 +126,8 @@
runner._options.world_leaks = False
test = 'failures/expected/reftest.html'
leak_test = 'failures/expected/leak.html'
- expectations = TestExpectations(runner._port, tests=[test, leak_test])
+ timeout_test = 'failures/expected/timeout.html'
+ expectations = TestExpectations(runner._port, tests=[test, leak_test, timeout_test])
expectations.parse_all_expectations()
runner._expectations = expectations
@@ -148,6 +149,24 @@
self.assertEqual(1, runner._current_run_results.expected)
self.assertEqual(0, runner._current_run_results.unexpected)
+ runner._current_run_results = TestRunResults(expectations, 3)
+ result = TestResult(timeout_test, failures=[])
+ runner.update_summary_with_result(result)
+ self.assertEqual(0, runner._current_run_results.expected)
+ self.assertEqual(1, runner._current_run_results.unexpected)
+ result = TestResult(timeout_test, failures=[test_failures.FailureTextMismatch()])
+ runner.update_summary_with_result(result)
+ self.assertEqual(0, runner._current_run_results.expected)
+ self.assertEqual(2, runner._current_run_results.unexpected)
+ result = TestResult(timeout_test, failures=[])
+ runner.update_summary_with_result(result)
+ self.assertEqual(0, runner._current_run_results.expected)
+ self.assertEqual(3, runner._current_run_results.unexpected)
+ result = TestResult(timeout_test, failures=[])
+ runner.update_summary_with_result(result)
+ self.assertEqual(0, runner._current_run_results.expected)
+ self.assertEqual(4, runner._current_run_results.unexpected)
+
def test_servers_started(self):
def start_http_server():
Modified: trunk/Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py (284226 => 284227)
--- trunk/Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py 2021-10-15 02:02:14 UTC (rev 284226)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/models/test_run_results.py 2021-10-15 02:20:48 UTC (rev 284227)
@@ -101,7 +101,7 @@
if existing_result.type is new_result.type:
return
- self.tests_by_expectation[existing_result.type].remove(existing_result.test_name)
+ self.tests_by_expectation[existing_result.type].discard(existing_result.test_name)
self.tests_by_expectation[new_result.type].add(new_result.test_name)
had_failures = len(existing_result.failures) > 0
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
