On 17/10/2018 01:23, Emilio Cobos Álvarez wrote:
Hi Philip,

Do you know how do reftests run in order to get that data?

I'm particularly curious about this Firefox-only failure:


It passes both on our automation and locally. I'm curious because I was the author of that test (whoops) and the Firefox fix (bug 1449010).

Does it use the same mechanism than our automation to wait for image decodes and such? Is there any way to see the test images?

It's using the same harness as we use in gecko, so it should be giving the same results, but of course it's possible that there's some difference in the configuration that could cause different results for some tests.

Unfortunately there isn't yet a way to see the images; because of the number of failures per run, and the number of runs, putting all the screenshots in the logs would be prohibitively large, but there is a plan to start uploading previously unseen screenshots to wpt.fyi [1]

Having said that the infrastructure is all containerised and it's possible to repeat the run locally with relatively little effort. I'm happy to help out with that if you like.

[1] https://github.com/web-platform-tests/wpt.fyi/issues/57
dev-platform mailing list

Reply via email to