On Fri, Jul 1, 2011 at 3:04 PM, Darin Adler <da...@apple.com> wrote: > On Jul 1, 2011, at 2:54 PM, Dirk Pranke wrote: > >> Does that apply to -expected.txt files in the base directories, or just >> platform-specific exceptions? > > Base directories. > > Expected files contain output reflecting the behavior of WebKit at the time > the test was checked in. The expected result when we re-run a test. Many > expected files contain text that says “FAIL” in them. The fact that these > expected results are not successes, but rather expected failures does not > seem to me to be a subtle point, but one of the basic things about how these > tests are set up. > >> I wonder how it is that I've been working (admittedly, mostly on tooling) in >> WebKit for more that two years and this is the first I'm hearing about this. > > I’m guessing it’s because you have been working on Chrome. > > The Chrome project came up with a different system for testing layered on top > of the original layout test machinery based on different concepts. I don’t > think anyone ever discussed that system with me; I was the one who created > the original layout test system, to help Dave Hyatt originally, and then > later the rest of the team started using it. > >> Are there reasons we [are] doing things this way[?] > > Sure. The idea of the layout test framework is to check if the code is still > behaving as it did when the test was created and last run; we want to detect > any changes in behavior that are not expected. When there are expected > changes in behavior, we change the contents of the expected results files.
I see, you're using "expected" in the sense of "this is what we expect to get", not "this is what is correct". > > It seems possibly helpful to augment the test system with editorial comments > about which tests show bugs that we’d want to fix. But I wouldn’t want to > stop running all regression tests where the output reflects the effects of a > bug or missing feature. > Of course, looking for changes in behavior is entirely the point of regression testing. Thank you for the history lesson :). -- Dirk _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
