On Sat, Oct 13, 2018, 09:17 Philip Jägenstedt <foo...@chromium.org> wrote:

> On Thu, Oct 11, 2018, 22:34 Boris Zbarsky <bzbar...@mit.edu> wrote:
>> On 10/11/18 4:22 PM, Philip Jägenstedt wrote:
>> > https://gist.github.com/foolip/a77c88e62aa3cfc461c2879f3e5d4855 is a
>> > list of tests that fail in Firefox Nightly, but pass in stable
>> > versions of Chrome, Edge and Safari.
>> Or more precisely have some sub-test that has that property, right?
> Right, since there's no way to link to a subtest, in those cases I've
> linked to the test and it might take some work to spot which subtest it
> was. If this is a problem I could improve the report.
> Thanks for filing the tracking bug, l hope there's some failures in here
> that point to problems that really affect web developers that can be fixed.

There's another crux worth mentioning. Tests can be definitely passing or
definitely failing, but then there are various crash/error/timeout/etc
results where the validity of the test is uncertain, or it's quite likely
to be a flake or infra issue. In my report I've been conservative and used
1 PASS + 3 FAIL as the criteria. Fiddling with these rules can reveal lots
more potential issues, and if you like I could provide reports on that too.

dev-platform mailing list

Reply via email to