Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 471453201b4f4770ce4eb2edb0539c8ff2cb4066
https://github.com/WebKit/WebKit/commit/471453201b4f4770ce4eb2edb0539c8ff2cb4066
Author: Carlos Alberto Lopez Perez <[email protected]>
Date: 2026-03-25 (Wed, 25 Mar 2026)
Changed paths:
M Tools/CISupport/ews-build/steps.py
M Tools/CISupport/ews-build/steps_unittest.py
Log Message:
-----------
[WKCI][WPE][GTK] Make the steps for GTK/WPE WK2 testing on the EWS less
likely to mark as good patches introducing new flakes.
https://bugs.webkit.org/show_bug.cgi?id=310315
Reviewed by Aakash Jain.
The current WK2 testing classes for GTK and WPE on the EWS were designed
when the layout test tree was frequently red, so they required a failure
to reproduce 10/10 times with the patch and 0/10 times without it before
marking a patch as bad. This makes it almost impossible to detect patches
that introduce new flakes.
Since the GTK and WPE trees are now consistently green, tighten the
failure detection: mark a patch as bad if a test fails once on the
first run with the patch and then passes all 10 retries without it.
This commit basically removes the step "repeat-failures-with-patch"
and adjusts the logic accordingly.
While at it, this also removes code duplication between the specialized
GTK/WPE class and the main WK2 EWS testing classes.
* Tools/CISupport/ews-build/steps.py:
(ParseByLineLogObserver.consumeLineGenerator):
(RunWebKitTests.run):
(RunWebKitTests._did_command_timed_out):
(RunWebKitTests.runCommand):
(RunWebKitTests._next_steps_for_platform):
(RunWebKitTests.evaluateCommand):
(RunWebKitTestsWithoutChange.evaluateCommand):
(RunWebKit1Tests.run):
(RunWebKitTestsRedTree._next_steps_for_platform):
(RunWebKitTestsRepeatFailuresWithoutChangeRedTree.setLayoutTestCommand):
(RunWebKitTestsRepeatFailuresWithoutChangeRedTree.runCommand):
(RunWebKitTestsWithoutChangeRedTree):
(AnalyzeLayoutTestsResultsRedTree.run):
(RunWebKitTestsRedTree._did_command_timed_out): Deleted.
(RunWebKitTestsRedTree.evaluateCommand): Deleted.
(RunWebKitTestsRepeatFailuresRedTree): Deleted.
(RunWebKitTestsRepeatFailuresRedTree.__init__): Deleted.
(RunWebKitTestsRepeatFailuresRedTree.setLayoutTestCommand): Deleted.
(RunWebKitTestsRepeatFailuresRedTree.evaluateCommand): Deleted.
(RunWebKitTestsRepeatFailuresRedTree.runCommand): Deleted.
(RunWebKitTestsRepeatFailuresRedTree.run): Deleted.
(RunWebKitTestsRepeatFailuresWithoutChangeRedTree.run): Deleted.
(RunWebKitTestsWithoutChangeRedTree.evaluateCommand): Deleted.
* Tools/CISupport/ews-build/steps_unittest.py:
(TestRunWebKitTestsRedTree.test_flakies_with_fail_retcode_then_go_to_analyze_results):
(TestRunWebKitTestsRedTree):
(TestRunWebKitTestsRedTree.test_flakies_with_warning_retcode_then_finish):
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree):
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree.configureStep):
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree.test_success):
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree.test_success_tests_names_with_shell_conflictive_chars):
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree.test_set_properties_when_executed_scope_this_class):
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree.test_run_ends_then_go_to_analyze):
(TestAnalyzeLayoutTestsResultsRedTree.test_failure_introduced_by_change_clean_tree_green):
(TestAnalyzeLayoutTestsResultsRedTree.test_failure_introduced_by_change_clean_tree_red):
(TestAnalyzeLayoutTestsResultsRedTree.test_pre_existent_failures):
(TestAnalyzeLayoutTestsResultsRedTree.test_pre_existent_flakies):
(TestAnalyzeLayoutTestsResultsRedTree.test_first_step_timeouts):
(TestAnalyzeLayoutTestsResultsRedTree.test_first_step_timeouts_last_try):
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_without_change_exits_early_error):
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_without_change_success):
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_without_change_warnings):
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_without_change_failure):
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_with_change_timeouts):
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_without_change_unexpected_error):
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_without_change_timeouts):
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_with_change_timeouts_and_without_change_timeouts):
(TestAnalyzeLayoutTestsResultsRedTree.test_retry_third_time):
(TestAnalyzeLayoutTestsResultsRedTree.test_retry_finish):
(TestRunWebKitTestsRedTree.test_flakies_but_no_failures_then_go_to_analyze_results):
Deleted.
(TestRunWebKitTestsRepeatFailuresRedTree): Deleted.
(TestRunWebKitTestsRepeatFailuresRedTree.setUp): Deleted.
(TestRunWebKitTestsRepeatFailuresRedTree.tearDown): Deleted.
(TestRunWebKitTestsRepeatFailuresRedTree.configureStep): Deleted.
(TestRunWebKitTestsRepeatFailuresRedTree.test_success): Deleted.
(TestRunWebKitTestsRepeatFailuresRedTree.test_success_tests_names_with_shell_conflictive_chars):
Deleted.
(TestRunWebKitTestsRepeatFailuresRedTree.test_set_properties_when_executed_scope_this_class):
Deleted.
(TestRunWebKitTestsRepeatFailuresRedTree.test_last_run_with_patch_ends_with_list_of_failing_tests_then_schedule_update_libs_and_test_without_patch):
Deleted.
(TestRunWebKitTestsRepeatFailuresRedTree.test_last_run_with_patch_ends_with_no_failing_tests_then_go_to_analyze):
Deleted.
(TestRunWebKitTestsRepeatFailuresWithoutChangeRedTree.test_step_with_change_did_timeout):
Deleted.
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_with_change_exits_early_error):
Deleted.
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_with_change_pass):
Deleted.
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_with_change_warnings):
Deleted.
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_with_change_error_with_flakies):
Deleted.
(TestAnalyzeLayoutTestsResultsRedTree.test_step_retry_with_change_unexpected_error):
Deleted.
Canonical link: https://commits.webkit.org/309933@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
