new-run-webkit-tests does not sample tests when they timeout. timeout is a perfectly "reasonable" test expectation. new-run-webkit-tests is about running all the tests and making sure their behavior matches what we have in test_expectations.txt. This is unlike run-webkit-tests for which the only valid expectation is "PASS".
new-run-webkit-tests makes sense in a world where you want to run all the tests, but have no prayer of passing them all. I think most ports are in this boat. Possibly even the Mac port these days. new-run-webkit-tests "always run every test, no matter its expected outcome" philosophy enables a bunch of neat fringe benefits, including documenting and tracking flaky tests w/o having them break your builders or requiring skipping. :) -eric On Sat, Apr 10, 2010 at 7:49 PM, Alexey Proskuryakov <a...@webkit.org> wrote: > > 10.04.2010, в 17:44, Eric Seidel написал(а): > >> Yes, I think we should keep Chromium's default low timeout. Having >> such a low timeout allows new-run-webkit-tests to easily run all flaky >> tests every time, even ones which occasionally timeout. > > > Does new-run-webkit-tests not sample tests that time out? Sampling is so > slow that is easily outweighs the difference between 3 or 30 seconds that a > test can run for. > > - WBR, Alexey Proskuryakov > > _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
