On Tue, Jun 12, 2012 at 9:50 AM, Darin Adler <[email protected]> wrote: > > - Each time we change the tests we need to make sure we haven’t broken new > tests on various platforms and that we’ve updated the skipped or test > expectations files properly. >
- When rearranging tests that share the test shell JavaScript files we’d > likely need to update relative paths. Might also be nice to put that test > shell in a more logical location. For these 2 points, we probably need some script to move a test or a set of tests. > - Calling the entire directory that hosts our regression test suite > LayoutTests seems wrong; that name was apropos when most of the tests were > for layout issues, very early in the life of the test machinery. The last time we had this discussion, the consensus appeared to be renaming it to RegressionTests. Do people have different opinions at this time? - Imported test suites ought to be gathered together in a single directory > then grouped by where they were imported from by having a directory for > each source. > Are you suggesting that we have LayoutTests/W3C? - We want to continue the practice of keeping copies imported test suites > as close to the original as possible. In some cases in the past that meant > preserving incorrect imported tests with expected “failures” that weren’t > really failures. Often we’d put corrected copies elsewhere in the test tree > while waiting for or working to get the imported test to fixed at the > source. > > - Keeping the tests that require the HTTP server in a separate directory > still seems like it might serve a purpose, although there seems to be an > extra directory level in the test hierarchy there that is not helpful. > - Making JavaScript-only tests runnable without WebKit’s involvement would > be a welcome feature; great for anyone working on JavaScriptCore. That > means we’d want to be sure to put those into their own directories. Ideally > that would eventually be coupled by a project to move the Mozilla test > suite currently in JavaScriptCore into the main WebKit regression tests. > Another point. Related to Oliver's response: http://lists.webkit.org/pipermail/webkit-dev/2011-September/017872.html on September 8th, 2011, we have at least 15 test running scripts (the number has since increased :( ) at this point: 1. run-api-tests 2. run-binding-tests 3. run-chromium-webkit-unit-tests 4. run-gtk-tests 5. run-iexploder-tests 6. run-javascriptcore-tests 7. run-mangleme-tests 8. run-perf-tests 9. run-qtwebkit-tests 10. run-regexp-tests 11. run-test-webkit-api # what's the difference between this and run-api-tests? 12. test-webkit-scripts 13. run-webkit-tests 14. test-webkitpy 15. test-webkitperl And I find it really annoying that it's non-trivial to figure out which script runs which tests and where the test code is located. Some of these test runners appear to be port specific while others seem to be port-agnostic. Also, given that the vast majority of these test runners also run some sort of regression tests (e.g. run-api-tests; API tests certainly are regression tests for some APIs or so I hope), renaming LayoutTests to RegressionTests might be a little confusing because of that. - Ryosuke
_______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
