On Sat, Oct 13, 2018, 09:17 Philip Jägenstedt <foo...@chromium.org> wrote:
> On Thu, Oct 11, 2018, 22:34 Boris Zbarsky <bzbar...@mit.edu> wrote: > >> On 10/11/18 4:22 PM, Philip Jägenstedt wrote: >> > https://gist.github.com/foolip/a77c88e62aa3cfc461c2879f3e5d4855 is a >> > list of tests that fail in Firefox Nightly, but pass in stable >> > versions of Chrome, Edge and Safari. >> >> Or more precisely have some sub-test that has that property, right? >> > > Right, since there's no way to link to a subtest, in those cases I've > linked to the test and it might take some work to spot which subtest it > was. If this is a problem I could improve the report. > > Thanks for filing the tracking bug, l hope there's some failures in here > that point to problems that really affect web developers that can be fixed. > There's another crux worth mentioning. Tests can be definitely passing or definitely failing, but then there are various crash/error/timeout/etc results where the validity of the test is uncertain, or it's quite likely to be a flake or infra issue. In my report I've been conservative and used 1 PASS + 3 FAIL as the criteria. Fiddling with these rules can reveal lots more potential issues, and if you like I could provide reports on that too. > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
