Good questions! I'm assuming the TIMEOUT results are caused by running
the tests in a non-release build, which is known to have significant
performance problems that cause timeouts in tests. This is not an ideal
state of affairs, obviously.
It's not clear to me what you mean by "full test suite", which is
usually what I would call `./mach test-wpt`.
As for when to run tests, I try to limit the tests I run to directories
that should be directly affected by the code I'm changing. This is
helped by considering that the web-platform-tests are basically
organized according to specification/subsection (html, dom,
xmlhttprequest, etc.) with some exceptions (webstorage, websockets,
etc.), as are the CSSWG tests. People working on layout code tend to
also run the tests in `./mach test-ref` and `./mach test-wpt
tests/wpt/mozilla/tests/css` as smoketests, and our build machines
uncover any remaining issues in the full suite when attempting to merge.
Cheers,
Josh
On 2015-10-20 10:06 PM, Gilles Leblanc wrote:
Hello,
I have some questions regarding the WPT tests. What is the proper way to
run those tests ?
Running :
./mach test-wpt --processes=8
Gives me the following results (on a branch with some modifications from
master):
Ran 207771 tests (3508 parents, 204263 subtests)
Expected results: 207060
Unexpected results: 12 (FAIL: 1, TIMEOUT: 11)
Skipped: 699
While running the full test suite:
Ran 6951 tests (6686 parents, 265 subtests)
Expected results: 6923
Unexpected results: 18 (FAIL: 1, PASS: 10, TIMEOUT: 7)
Skipped: 10
What puzzles me is that I have no PASS results with the larger test base.
Also running these takes a really long time. Is there a guideline for
when/how to run them?
Thanks,
Gilles
_______________________________________________
dev-servo mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-servo
