Dave, Adam, and All,
Thanks for your helpful answers. I still have some more questions for you guys:
1- Dave said "Image test is required to capture expected results that a text based test can't". Can you please provide me with a path/name of one of these cases?
2- I actually need to know which of the layout test cases I'm adding requires Image test, so I can generate the PNGs and Checksums for these specific test cases (through the -p option of the "run-webkit-tests" script)- Or do you think that it's better to generate the PNGs and checksums for all the test cases I'm planning to add?
3- How is the PNGs comparison implemented for regression testing? (in the text based tests it's a string comparison how about the image based tests?)
4- What are the Checksum files used for?
5- So, please correct me if there's anything missing in the following steps: In order to test a WebKit based port other than Safari and Qt with WebKit's Layout tests I need to:
a- update the "run-webkit-tests" Perl script.
b- port the "DumpRenderTree" app.
c- regenerate the expected results for each of the layout tests
(knowing that the render tree dumps are different for each platform)
Thanks.
-Reem
From:"David D. Kilzer" [EMAIL PROTECTED]Reply-To:[EMAIL PROTECTED]To:Reem Yazigi [EMAIL PROTECTED]CC:webkit-dev@lists.webkit.org, [EMAIL PROTECTED]Subject:Re: [webkit-dev] Adding New Layout TestsDate:Tue, 29 May 2007 18:14:23 -0700 (PDT) 1- Why not all of the layout tests are working on Qt? Aren't they supposed totest the core?Because the Qt port is still a work-in-progress, although it is progressingrapidly! So, why do they all work on Safari but not on Qt?Because they were written first for the Safari port. 2- Why is the render tree dumps different for each platform?They should be the
same, but invariably differences in fonts (and thus fontmetrics) may cause the pages to be laid out slightly differently.I'm not sureif there are any other differences; in "theory", they should be the same forall platforms. 3- Why would we generate PNGs and checksum files? Why would we use them andfor what?Sometimes an "image test" is required to capture expected results that atext-based test can't (such as rendering of graphic images in web pages withproper colors).You are correct in thinking that any test that does not need an image testshould be changed to a text-only test, but there are a number of original testswhere this was not done.(Patches to fix this are welcome, however.) 4- How can we make sure the expected results
files that get automaticallygenerated for new tests are correct?Visual inspection by a human.If this is a new port, you should probably writesome kind of script that compares "known good" results with a new port'sresults using some kind of "confidence interval" logic.I think the Qt guysmay have written something like this already for their port?I don't see it inWebKitTools/Scripts, though.DaveReem Yazigi [EMAIL PROTECTED] wrote: Dave, Kirby and all, Thanks for the useful answers and documentation. I have some more questionsthat I need answers for before I start adding new test cases: 1- Why not all of the layout tests are working on Qt?
Aren't they supposed totest the core? So, why do they all work on Safari but not on Qt? 2- Why is the render tree dumps different for each platform? 3- Why would we generate PNGs and checksum files? Why would we use them andfor what? 4- How can we make sure the expected results files that get automaticallygenerated for new tests are correct? Thanks, Reem. "David D. Kilzer" [EMAIL PROTECTED] wrote:Reem Yazigi [EMAIL PROTECTED] wrote: 1- Are the expected-results files browser dependent? The expected results files are simply text, but
there are two basictypes:render tree dumps and text dumps.The render tree dumps will bedifferent foreach platform, while the text dumps should be the same (or much closer). (are the ones "run-webkit-tests" creates for Qt identicalto the ones it creates for Safari? No. Do I have to change that if I want to run the tests for gdk?) Yes, but run-webkit-tests has a way to save results in a differentdirectorystructure, which is what Qt is doing. 2- When I run the
"run-webkit-tests" for Qt, 238 test casesget skipped and only about 4100 get executed! I believe they'reabout 7700 on Safari!! Why is it that different? Because not all of the tests are working on Qt yet. 3- Is there any parameter that needs to be passed to the"run-webkit-tests" script or to the "DumpRenderTree" in orderto specify the browser's window dimensions?? I'm actually gettingdifferent results when I run "run-webkit-tests" for Qt ondifferent devices!! I don't know about Qt, but the dimensions
are hard-coded in theDumpRenderTree.m source file for the native Mac version (800x600). I couldn't access the IRC channel!! Is it still available? The #webkit
