One other difference I should point out: Skipped lists "cascade" through the normal results fallback mechanisms (if it ever doesn't exactly match ORWT, that's a bug!). For example, Mac uses platform/mac-snowleopard/Skipped, combined with platform/mac/Skipped. http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py#L288
test_expectations do not. test_expectations only use your "base port name" (ignoring the version). For example, platform/mac/test_expectations.txt. http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py#L73 NRWT uses both! It will read in all the port's Skipped files, covert them to SKIP text_expectations, and add them to your test_expectations file. http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py#L309 For better or worse, NRWT will error out, if you have duplicates in your test_expectations file, including duplicates between your test_expectations file and your Skipped lists. Hopefully this makes sense. We can of course change all this behavior if folks need in the future. -eric On Wed, Jul 6, 2011 at 8:58 AM, Adam Barth <[email protected]> wrote: > I'm not sure we've quite figured that out yet. NRWT supports both > Skipped lists and test_expectations.txt, which is a more expressive > (but also more complex) version of Skipped lists. IMHO, we should > wait for the dust to settle on the transition before changing our > practices. > > Adam > > > On Wed, Jul 6, 2011 at 8:53 AM, Adam Roben <[email protected]> wrote: >> Now that more and more ports are switching to NRWT, it would be great for >> someone to explain what the best practices are for dealing with failing and >> flaky tests. >> >> -Adam > _______________________________________________ > webkit-dev mailing list > [email protected] > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
