On Tue, Jun 12, 2012 at 9:50 AM, Darin Adler <da...@apple.com> wrote:
> On Jun 11, 2012, at 7:33 PM, Mike Lawther wrote: > > > Does having the 'fast' directory still serve a useful purpose? > > Not sure it does. > > The original intent was to put more extensive complete tests that were > slow but had greater coverage outside the “fast” directory, so that you > could run the majority of the most useful tests in a few seconds. > > Times to run tests have increased so much, despite the parallel test > runner, that it’s no longer a useful distinction. > > A new logical organization for tests would be welcome, and it’s a big > project where I am not sure we have a clear enough plan to start on it > incrementally. Here are a few considerations: > > - Moving test files around could cause some huge churn, with some > super-slow check-outs for everyone working on the project and many old > patches potentially needing to be rebased. This is especially true when > image results are involved. > > - Each time we change the tests we need to make sure we haven’t broken new > tests on various platforms and that we’ve updated the skipped or test > expectations files properly. > If we could get more EWS bots running the tests (currently only Chromium Linux does), that would make this a lot easier. It would also improve general development on WebKit. Fewer cases of people being surprised by broken layout tests when a patch gets committed. One more item I'd add to this list: -Replace js-test-post.js with an onload handler in js-test-pre.js so we can get rid of that file entirely. > - Calling the entire directory that hosts our regression test suite > LayoutTests seems wrong; that name was apropos when most of the tests were > for layout issues, very early in the life of the test machinery. > > - Imported test suites ought to be gathered together in a single directory > then grouped by where they were imported from by having a directory for > each source. > > - We want to continue the practice of keeping copies imported test suites > as close to the original as possible. In some cases in the past that meant > preserving incorrect imported tests with expected “failures” that weren’t > really failures. Often we’d put corrected copies elsewhere in the test tree > while waiting for or working to get the imported test to fixed at the > source. > > - Keeping the tests that require the HTTP server in a separate directory > still seems like it might serve a purpose, although there seems to be an > extra directory level in the test hierarchy there that is not helpful. > > - When rearranging tests that share the test shell JavaScript files we’d > likely need to update relative paths. Might also be nice to put that test > shell in a more logical location. > > - Making JavaScript-only tests runnable without WebKit’s involvement would > be a welcome feature; great for anyone working on JavaScriptCore. That > means we’d want to be sure to put those into their own directories. Ideally > that would eventually be coupled by a project to move the Mozilla test > suite currently in JavaScriptCore into the main WebKit regression tests. > > -- Darin > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev >
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
