Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 118e5ea7dccdc5c700252646be5951025c4e5306
https://github.com/WebKit/WebKit/commit/118e5ea7dccdc5c700252646be5951025c4e5306
Author: Issac Roy <[email protected]>
Date: 2026-09-03 (Thu, 03 Sep 2026)
Changed paths:
M Tools/CISupport/ews-build/results_db.py
M Tools/CISupport/ews-build/steps.py
M Tools/CISupport/ews-build/steps_unittest.py
Log Message:
-----------
[EWS] A clean-tree flake conviction can rest on a single observation from the
same author
https://bugs.webkit.org/show_bug.cgi?id=323115
rdar://186374059
Reviewed by Aakash Jain.
EWS excuses a failing test when the results database already calls it flaky. A
`WithinStepCleanTree` row convicts on its own, with no threshold, while a
dirty-tree verdict demands two pull requests and two authors and a
between-builds verdict three and two. Nothing compared that evidence against the
change being judged, so a pull request's own earlier build could excuse its own
failure and the test was dropped before it ever ran against a clean tree.
GTK-WK2-Tests-EWS build 153043 excused a crash recorded by build 153004, the
same pull request by the same author; the test then failed 9 of 14 runs on main
after it landed, and 0 of 35 before.
`_rows_by_type` buckets the rows by `flaky_type` in one pass and drops those
that are not independent evidence: a row goes when its `pr_number` is the pull
request being judged, or when every author it records is one of the change's
own.
Both axes are needed: the merge queue records the login of whoever applied the
label rather than the submitter, so the author check alone lets a PR's own row
through. `authors` comes from the `owners` property that `results_db_details`
already writes into each row's `details`, so there is no new source of truth. A
build where neither GitHub step ran leaves `owners` holding buildbot's blamelist
of commit authors rather than GitHub logins; the comparison then matches nothing
and conviction proceeds as it does today.
A `WithinStepCleanTree` row is exempt from that filtering. It is recorded after
the change is reverted and recompiled, so the pull request and authors on it
name the build that ran rather than a party with a stake in the verdict, and
dropping it for naming the change under test would discard the one observation
that already excludes the change as the cause. Independence there is a matter of
how many builds saw it, not who owned them: `_convict` takes a `builds_needed`
counted through `evidence.build_urls`, and `BUILDS_FOR_CLEAN_TREE_FLAKE` is 2.
All three thresholds default to 0, so the two existing call sites are unchanged.
Below that threshold the rule logs what it saw and falls through to the
dirty-tree rule instead of returning, so a test with one clean-tree row and
enough dirty-tree evidence is still convicted. The two rules read disjoint
buckets, so no row counts toward both, and `INCLUDED_FLAKY_VERDICTS` already
carries either verdict.
`intra_build_evidence` is now taken from the rows that survived that filtering
rather than the raw query, since a between-builds verdict whose only
within-build rows came from the change itself is exactly what the warning it
feeds exists to report. A clean-tree row the change wrote itself does count as
within-build evidence, so a `BetweenBuilds` verdict recorded alongside one is no
longer reported as `results-db_flaky_unsupported`.
`AnalyzeAPITestsResults` reads the database for the same reason and drops a
failure on the same verdicts, so it passes the same identity. `APITestsFactory`
derives from `TestFactory`, whose base adds `ValidateChange`, so `owners` and
`github.number` reach an API build by the same route as a layout build, and the
blamelist fallback above applies unchanged.
* Tools/CISupport/ews-build/results_db.py:
(ResultsDatabase):
(ResultsDatabase._convict):
(ResultsDatabase._rows_by_type):
(ResultsDatabase._rows_by_type.written_by_another_pull_request):
(ResultsDatabase._rows_by_type.written_by_others):
(ResultsDatabase._is_intra_build_flake):
(ResultsDatabase._is_inter_build_flake):
(ResultsDatabase.flaky_verdicts_for):
(ResultsDatabase._rows_by_flaky_type): Deleted.
(ResultsDatabase.flaky_verdicts_for.logger): Deleted.
* Tools/CISupport/ews-build/steps.py:
(RunWebKitTests.filter_failures_using_results_db):
(AnalyzeAPITestsResults.flaky_new_failures_using_results_db):
* Tools/CISupport/ews-build/steps_unittest.py:
Canonical link: https://commits.webkit.org/320410@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
