Dave, Adam, and All,
Thanks for your helpful answers. I still have some more questions for you guys:
1- Dave said "Image test is required to capture expected results that a text based test can't". Can you please provide me with a path/name of one of these cases?
2- I actually need to know which of the layout test cases I'm adding requires Image test, so I can generate the PNGs and Checksums for these specific test cases (through the -p option of the "run-webkit-tests" script)- Or do you think that it's better to generate the PNGs and checksums for all the test cases I'm planning to add?
3- How is the PNGs comparison implemented for regression testing? (in the text based tests it's a string comparison how about the image based tests?)
4- What are the Checksum files used for?
5- So, please correct me if there's anything missing in the following steps: In order to test a WebKit based port other than Safari and Qt with WebKit's Layout tests I need to:
a- update the "run-webkit-tests" Perl script.
b- port the "DumpRenderTree" app.
c- regenerate the expected results for each of the layout tests
(knowing that the render tree dumps are different for each platform)
Thanks.
-Reem
From: "David D. Kilzer" <[EMAIL PROTECTED]>Reply-To: [EMAIL PROTECTED]To: Reem Yazigi <[EMAIL PROTECTED]>CC: webkit-dev@lists.webkit.org, [EMAIL PROTECTED]Subject: Re: [webkit-dev] Adding New Layout TestsDate: Tue, 29 May 2007 18:14:23 -0700 (PDT)> > 1- Why not all of the layout tests are working on Qt? Aren't they supposed to>test> > the core?>>Because the Qt port is still a work-in-progress, although it is progressing>rapidly!>> > So, why do they all work on Safari but not on Qt?>>Because they were written first for the Safari port.>> > 2- Why is the render tree dumps different for each platform?>>They should be the
same, but invariably differences in fonts (and thus font>metrics) may cause the pages to be laid out slightly differently. I'm not sure>if there are any other differences; in "theory", they should be the same for>all platforms.>> > 3- Why would we generate PNGs and checksum files? Why would we use them and>for what?>>Sometimes an "image test" is required to capture expected results that a>text-based test can't (such as rendering of graphic images in web pages with>proper colors).>>You are correct in thinking that any test that does not need an image test>should be changed to a text-only test, but there are a number of original tests>where this was not done. (Patches to fix this are welcome, however.)>> > 4- How can we make sure the expected results
files that get automatically>generated> > for new tests are correct?>>Visual inspection by a human. If this is a new port, you should probably write>some kind of script that compares "known good" results with a new port's>results using some kind of "confidence interval" logic. I think the Qt guys>may have written something like this already for their port? I don't see it in>WebKitTools/Scripts, though.>>Dave>>>Reem Yazigi <[EMAIL PROTECTED]> wrote:>> > Dave, Kirby and all,> >> > Thanks for the useful answers and documentation. I have some more questions>that I> > need answers for before I start adding new test cases:> >> > 1- Why not all of the layout tests are working on Qt?
Aren't they supposed to>test> > the core? So, why do they all work on Safari but not on Qt?> >> > 2- Why is the render tree dumps different for each platform?> >> > 3- Why would we generate PNGs and checksum files? Why would we use them and>for what?> >> > 4- How can we make sure the expected results files that get automatically>generated> > for new tests are correct?> >> > Thanks,> >> > Reem.> >> > "David D. Kilzer" <[EMAIL PROTECTED]> wrote:> > > > Reem Yazigi <[EMAIL PROTECTED]> wrote:> > > >> > > > > 1- Are the expected-results files browser dependent?> > > >> > > > The expected results files are simply text, but
there are two basic>types:> > > > render tree dumps and text dumps. The render tree dumps will be>different for> > > > each platform, while the text dumps should be the same (or much closer).> > > >> > > > > (are the ones "run-webkit-tests" creates for Qt identical> > > > > to the ones it creates for Safari?> > > >> > > > No.> > > >> > > > > Do I have to change that if I want to run the tests for gdk?)> > > >> > > > Yes, but run-webkit-tests has a way to save results in a different>directory> > > > structure, which is what Qt is doing.> > > >> > > > > 2- When I run the
"run-webkit-tests" for Qt, 238 test cases> > > > > get skipped and only about 4100 get executed! I believe they're> > > > > about 7700 on Safari!! Why is it that different?> > > >> > > > Because not all of the tests are working on Qt yet.> > > >> > > > > 3- Is there any parameter that needs to be passed to the> > > > > "run-webkit-tests" script or to the "DumpRenderTree" in order> > > > > to specify the browser's window dimensions?? I'm actually getting> > > >
