Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: b3f6dc92c6a45dbd5f4fcec996c551e319ffe678
https://github.com/WebKit/WebKit/commit/b3f6dc92c6a45dbd5f4fcec996c551e319ffe678
Author: Sam Sneddon <[email protected]>
Date: 2026-08-22 (Sat, 22 Aug 2026)
Changed paths:
M Tools/Scripts/libraries/webkitcorepy/webkitcorepy/testing/test_runner.py
A
Tools/Scripts/libraries/webkitcorepy/webkitcorepy/tests/testing/__init__.py
A
Tools/Scripts/libraries/webkitcorepy/webkitcorepy/tests/testing/test_runner_unittest.py
M Tools/Scripts/webkitpy/test/main.py
M Tools/Scripts/webkitpy/test/printer.py
A Tools/Scripts/webkitpy/test/printer_unittest.py
M Tools/Scripts/webkitpy/test/runner.py
M Tools/Scripts/webkitpy/test/runner_unittest.py
Log Message:
-----------
[webkitpy] [webkitcorepy] Expected failures / unexpected successes not
properly handled
https://bugs.webkit.org/show_bug.cgi?id=309834
rdar://172412849
Reviewed by Elliott Williams.
The test runners did not correctly handle Python's @unittest.expectedFailure
decorator. Expected failures were reported as "passed", and unexpected
successes (where a test marked as expected-to-fail actually passes) were
conflated with regular failures without being clearly identified.
Track expected failures and unexpected successes as distinct outcomes
throughout the pipeline, instead of conflating them with passes and
failures.
* Tools/Scripts/libraries/webkitcorepy/webkitcorepy/testing/test_runner.py:
(TestRunner.run): Report expected failures as "expected failure" instead of
"passed", and treat unexpected successes as failures.
* Tools/Scripts/libraries/webkitcorepy/webkitcorepy/tests/testing/__init__.py:
Added.
*
Tools/Scripts/libraries/webkitcorepy/webkitcorepy/tests/testing/test_runner_unittest.py:
Added.
(_Test_TestRunner):
(_Test_TestRunner._test_pass):
(_Test_TestRunner._test_fail):
(_Test_TestRunner._test_error):
(_Test_TestRunner._test_skip):
(_Test_TestRunner._test_expected_failure):
(_Test_TestRunner._test_unexpected_success):
(_Test_TestRunner.id):
(StubTestRunner):
(StubTestRunner.__init__):
(StubTestRunner.tests):
(StubTestRunner.run_test):
(TestRunnerTest):
(TestRunnerTest._normalize):
(TestRunnerTest._run):
(TestRunnerTest.test_passed):
(TestRunnerTest.test_failed):
(TestRunnerTest.test_errored):
(TestRunnerTest.test_skipped):
(TestRunnerTest.test_unexpected_success_causes_failure):
(TestRunnerTest.test_expected_failure_returns_success):
* Tools/Scripts/webkitpy/test/main.py:
(Tester._run_tests): Include expected/actual values for expected failures
and unexpected successes in uploaded test results.
* Tools/Scripts/webkitpy/test/printer.py:
(Printer.print_finished_test): Accept expected_failures and
unexpected_successes, so unexpected successes are shown as failures and
expected failures get their own summary suffix.
* Tools/Scripts/webkitpy/test/printer_unittest.py: Added.
(PrinterTest):
(PrinterTest.setUp):
(PrinterTest._make_printer):
(PrinterTest._drive):
(PrinterTest.test_passed):
(PrinterTest.test_failed):
(PrinterTest.test_errored):
(PrinterTest.test_unexpected_success):
(PrinterTest.test_expected_failure):
(PrinterTest.test_failed_with_expected_failures_list):
* Tools/Scripts/webkitpy/test/runner.py:
(Runner.__init__): Track expected failures and unexpected successes.
(Runner.handle): Ditto.
(_Worker.handle): Compute expected_failures and unexpected_successes
separately instead of folding unexpected successes into failures.
* Tools/Scripts/webkitpy/test/runner_unittest.py:
(FakeModuleSuite.run): Support 'x' (expected failure) and 'u' (unexpected
success) fake results.
(RunnerTest.test_run_expected_failures):
Canonical link: https://commits.webkit.org/319652@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
