Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: ab9b56ad6cc1a19caccdfb252dc3408030f237c6
https://github.com/WebKit/WebKit/commit/ab9b56ad6cc1a19caccdfb252dc3408030f237c6
Author: Issac Roy <[email protected]>
Date: 2026-09-04 (Fri, 04 Sep 2026)
Changed paths:
M Tools/CISupport/ews-build/results_db.py
M Tools/CISupport/ews-build/steps.py
M Tools/CISupport/ews-build/steps_unittest.py
Log Message:
-----------
[EWS] Each results-database test step carries its own copy of the shared
reporting behaviour
https://bugs.webkit.org/show_bug.cgi?id=323387
rdar://186616569
Reviewed by Aakash Jain.
The layout, API and JSC test steps all report to the results database
through `ResultsDBReportMixin`, but each carried its own log name, its
own copy of the query constants, its own copy of the configuration a
query is built from, its own copy of the lookup deciding which commit to
ask about, and its own copy of the classifier deciding which new
failures the database already knew were flaky. A change to any of it had
to be made three times, and a step that drifted from the others drew no
complaint. Hoist the lookup as `resolve_identifier_for_results_db`, the
classifier as `pre_existing_flakes_using_results_db`, the two sentences
naming ignored evidence as `results_db_ignore_message` and
`results_db_ignore_counts_message`, and `NUM_FAILURES_TO_DISPLAY`, and
drop the duplicate constants. `RunWebKitTests` keeps its own
`MAX_FAILURES_TO_CHECK_RESULTS_DB`, since it asks about 60 failures
rather than the shared 50.
The classifier names the properties it writes from `prefix`, so layout
gains `prefix = 'first_run_'` on `RunWebKitTests` and `'second_run_'` on
`ReRunWebKitTests` in place of the names it spelled out. Every
`results-db_` property keeps the name it had, which matters because
builds are ingested by keying on those strings, so a rename loses data
rather than failing. The `pre_existing` property stays written at each
call site, since the API steps build that one name from `suffix` and
folding it in would rename four of them.
Layout excused a `BetweenBuilds` verdict that carried no
`intra_build_evidence`, and the shared classifier does not, so layout is
tightened to agree with API and JSC. A test failing only in that shape
is now blamed on the change rather than ignored.
Some text a reader sees changes with the sharing. The JSC log line gains
the trailing newline it was missing and reads "failing tests" like the
others rather than "failing JSC tests". The API step's ignored-evidence
line reads "Ignored pre-existing failures" rather than "Ignored
pre-existing failure", and can now name ignored flakes as well.
The two ignored-evidence sentences change shape as well. A `; ` joined the
two halves under one leading "Ignored", so the flaky half read as a
continuation of the failure half; each half now carries its own "Ignored"
and they are joined by a newline, which means `build_summary` and
`descriptionDone` can hold two lines where they held one. "flaky tests"
becomes "pre-existing flakes", the name the attribute and the property
already use, and the trailing "based on results-db" goes from all three
sentences — a reader who cares which database answered is already reading
the results-db log.
`preexisting_failures_in_results_db` and `flaky_failures_in_results_db`
become `pre_existing_failures_in_results_db` and
`pre_existing_flakes_in_results_db`. A flake found this way is
pre-existing in the sense the failures are, and `pre_existing` with the
underscore is the spelling `is_test_pre_existing_failure` and the
property names already use.
`FindUnexpectedStaticAnalyzerResults` already declared
`results_db_log_name` and `suite` but not the mixin that defines them,
which is why it could not reach the shared query configuration until it
listed the mixin among its bases.
A row the results database returns did not know which test it described,
so the lines saying which evidence was ignored — whether a row was
recorded by the change's own pull request, carried no `flaky_type`, or
fell below the clean-tree bar — named no test at all. A build with
several failing tests therefore produced a log whose lines could not be
attributed to any of them. Carry the test name on `EWSRow` and print it
on each of those lines.
* Tools/CISupport/ews-build/results_db.py:
(EWSRow):
(EWSRow.from_json):
(ResultsDatabase):
(ResultsDatabase._is_intra_build_flake):
(ResultsDatabase._is_inter_build_flake):
* Tools/CISupport/ews-build/steps.py:
(ResultsDBReportMixin):
(ResultsDBReportMixin.resolve_identifier_for_results_db):
(RunJavaScriptCoreTests):
(RunJavaScriptCoreTests.filter_failures_using_results_db):
(RunWebKitTests):
(RunWebKitTests.filter_failures_using_results_db):
(RunAPITests):
(RunAPITests.filter_api_test_failures_using_results_db):
(AnalyzeAPITestsResults):
(ResultsDBReportMixin.results_db_ignore_message):
(ResultsDBReportMixin.results_db_ignore_counts_message):
(ResultsDBReportMixin.pre_existing_flakes_using_results_db):
(RunJavaScriptCoreTests.__init__):
(RunJavaScriptCoreTests.runCommand):
(RunJavaScriptCoreTests.evaluateCommand):
(RunJavaScriptCoreTests._check_for_pre_existing_failures):
(RunJavaScriptCoreTests.getResultSummary):
(AnalyzeJSCTestsResults):
(RunWebKitTests.__init__):
(RunWebKitTests.runCommand):
(RunWebKitTests.evaluateCommand):
(RunWebKitTests.getResultSummary):
(ReRunWebKitTests):
(ReRunWebKitTests.evaluateCommand):
(ReRunWebKitTests.runCommand):
(AnalyzeLayoutTestsResults):
(RunAPITests.__init__):
(RunAPITests.run):
(RunAPITests.analyze_failures_using_results_db):
(ReRunAPITests.doOnFailure):
(AnalyzeAPITestsResults.run):
(FindUnexpectedStaticAnalyzerResults):
(FindUnexpectedStaticAnalyzerResults.filter_results_using_results_db):
(ResultsDBReportMixin.flaky_new_failures_using_results_db): Deleted.
(RunJavaScriptCoreTests._check_for_preexisting_failures): Deleted.
* Tools/CISupport/ews-build/steps_unittest.py:
(TestFilterLayoutTestFailuresUsingResultsDB._check_order):
(TestFilterLayoutTestFailuresUsingResultsDB.test_a_verdict_in_the_included_set_removes_the_failure):
(TestFilterLayoutTestFailuresUsingResultsDB.test_a_verdict_outside_the_included_set_is_recorded_without_ignoring_the_failure):
(TestFilterLayoutTestFailuresUsingResultsDB):
(TestFilterLayoutTestFailuresUsingResultsDB.test_a_between_builds_verdict_with_no_intra_build_evidence_is_not_excused):
(TestFilterLayoutTestFailuresUsingResultsDB.test_a_test_with_no_verdict_is_not_treated_as_sound):
(TestFilterLayoutTestFailuresUsingResultsDB.test_ignoring_every_failure_sets_the_property_set_build_summary_reads):
(TestJSCTestStepsReportAsTheyRun.configureRunStep.fake_filter):
(TestLayoutTestStepsReportAsTheyRun):
(TestLayoutTestStepsReportAsTheyRun.configureStep.fake_filter):
(TestLayoutTestStepsReportAsTheyRun.test_the_first_run_exports_the_verdicts_it_relied_on):
(TestLayoutTestStepsReportAsTheyRun.test_the_rerun_exports_the_verdicts_it_relied_on_under_its_own_prefix):
(TestLayoutTestStepsReportAsTheyRun.test_the_first_run_exports_the_verdicts_it_relied_on.fake_filter):
Deleted.
Canonical link: https://commits.webkit.org/320501@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
