Dave, Adam, and All,
Thanks for your helpful answers. I still have some more questions for you guys:
1- Dave said "Image test is required to capture expected results that a text based test can't". Can you please provide me with a path/name of one of these cases?
2- I actually need to know which of the layout test cases I'm adding requires Image test, so I can generate the PNGs and Checksums for these specific test cases (through the -p option of the "run-webkit-tests" script)- Or do you think that it's better to generate the PNGs and checksums for all the test cases I'm planning to add?
3- How is the PNGs comparison implemented for regression testing? (in the text based tests it's a string comparison how about the image based tests?)
4- What are the Checksum files used for?
5- So, please correct me if there's anything missing in the following steps: In order to test a WebKit based port other than Safari and Qt with WebKit's Layout tests I need to:
a- update the "run-webkit-tests" Perl script.
b- port the "DumpRenderTree" app.
c- regenerate the expected results for each of the layout tests
(knowing that the render tree dumps are different for each platform)
Thanks.
-Reem
From: "David D. Kilzer" <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
To: Reem Yazigi <[EMAIL PROTECTED]>
CC: webkit-dev@lists.webkit.org, [EMAIL PROTECTED]
Subject: Re: [webkit-dev] Adding New Layout Tests
Date: Tue, 29 May 2007 18:14:23 -0700 (PDT)
> > 1- Why not all of the layout tests are working on Qt? Aren't they supposed to
>test
> > the core?
>
>Because the Qt port is still a work-in-progress, although it is progressing
>rapidly!
>
> > So, why do they all work on Safari but not on Qt?
>
>Because they were written first for the Safari port.
>
> > 2- Why is the render tree dumps different for each platform?
>
>They should be the same, but invariably differences in fonts (and thus font
>metrics) may cause the pages to be laid out slightly differently. I'm not sure
>if there are any other differences; in "theory", they should be the same for
>all platforms.
>
> > 3- Why would we generate PNGs and checksum files? Why would we use them and
>for what?
>
>Sometimes an "image test" is required to capture expected results that a
>text-based test can't (such as rendering of graphic images in web pages with
>proper colors).
>
>You are correct in thinking that any test that does not need an image test
>should be changed to a text-only test, but there are a number of original tests
>where this was not done. (Patches to fix this are welcome, however.)
>
> > 4- How can we make sure the expected results files that get automatically
>generated
> > for new tests are correct?
>
>Visual inspection by a human. If this is a new port, you should probably write
>some kind of script that compares "known good" results with a new port's
>results using some kind of "confidence interval" logic. I think the Qt guys
>may have written something like this already for their port? I don't see it in
>WebKitTools/Scripts, though.
>
>Dave
>
>
>Reem Yazigi <[EMAIL PROTECTED]> wrote:
>
> > Dave, Kirby and all,
> >
> > Thanks for the useful answers and documentation. I have some more questions
>that I
> > need answers for before I start adding new test cases:
> >
> > 1- Why not all of the layout tests are working on Qt? Aren't they supposed to
>test
> > the core? So, why do they all work on Safari but not on Qt?
> >
> > 2- Why is the render tree dumps different for each platform?
> >
> > 3- Why would we generate PNGs and checksum files? Why would we use them and
>for what?
> >
> > 4- How can we make sure the expected results files that get automatically
>generated
> > for new tests are correct?
> >
> > Thanks,
> >
> > Reem.
> >
> > "David D. Kilzer" <[EMAIL PROTECTED]> wrote:
> > > > Reem Yazigi <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > 1- Are the expected-results files browser dependent?
> > > >
> > > > The expected results files are simply text, but there are two basic
>types:
> > > > render tree dumps and text dumps. The render tree dumps will be
>different for
> > > > each platform, while the text dumps should be the same (or much closer).
> > > >
> > > > > (are the ones "run-webkit-tests" creates for Qt identical
> > > > > to the ones it creates for Safari?
> > > >
> > > > No.
> > > >
> > > > > Do I have to change that if I want to run the tests for gdk?)
> > > >
> > > > Yes, but run-webkit-tests has a way to save results in a different
>directory
> > > > structure, which is what Qt is doing.
> > > >
> > > > > 2- When I run the "run-webkit-tests" for Qt, 238 test cases
> > > > > get skipped and only about 4100 get executed! I believe they're
> > > > > about 7700 on Safari!! Why is it that different?
> > > >
> > > > Because not all of the tests are working on Qt yet.
> > > >
> > > > > 3- Is there any parameter that needs to be passed to the
> > > > > "run-webkit-tests" script or to the "DumpRenderTree" in order
> > > > > to specify the browser's window dimensions?? I'm actually getting
> > > > > different results when I run "run-webkit-tests" for Qt on
> > > > > different devices!!
> > > >
> > > > I don't know about Qt, but the dimensions are hard-coded in the
> > > > DumpRenderTree.m source file for the native Mac version (800x600).
> > > >
> > > > > I couldn't access the IRC channel!! Is it still available?
> > > >
> > > > The #webkit channel on irc.freenode.net? http://webkit.org/contact.html
> > > >
> > > > Dave
>
Like puzzles? Play free games & earn great prizes. Play Clink now.
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
