Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-05-21 Thread Adam Roben
On Feb 23, 2010, at 6:42 PM, Maciej Stachowiak wrote: The challenge with reftests is coming up with different markup that's supposed to render exactly the same, but for unrelated reasons. While it's possible, for example, to compare a table layout to absolute positioned boxes, for some

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-04-21 Thread Ojan Vafai
On Tue, Apr 13, 2010 at 1:49 PM, Simon Fraser simon.fra...@apple.comwrote: dumpAsText also turns off pixel results, which seems like a loss of useful testing. What's the plan there? The hope is that the dumpRenderTree tests will be sufficient testing for pixel results. On Tue, Apr 13, 2010

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-04-13 Thread Ojan Vafai
On Mon, Mar 1, 2010 at 7:53 PM, Maciej Stachowiak m...@apple.com wrote: On Mar 1, 2010, at 6:46 PM, Ojan Vafai wrote: On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak m...@apple.com wrote: On the one hand, it's good to shift the default. On the other hand, many render tree dumping tests do

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-04-13 Thread David Hyatt
There are also a fair number of directories that test rendering and layout, although they are usually obvious by their names... many of the subdirectories in fast cannot be converted to pure text tests (clipping, margin collapsing, backgrounds, etc.). dave On Apr 13, 2010, at 1:54 PM, Ojan

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-04-13 Thread Simon Fraser
On Apr 13, 2010, at 11:54 AM, Ojan Vafai wrote: On Mon, Mar 1, 2010 at 7:53 PM, Maciej Stachowiak m...@apple.com wrote: On Mar 1, 2010, at 6:46 PM, Ojan Vafai wrote: On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak m...@apple.com wrote: On the one hand, it's good to shift the default. On

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-04-13 Thread Dirk Pranke
On Tue, Apr 13, 2010 at 11:54 AM, Ojan Vafai o...@chromium.org wrote: On Mon, Mar 1, 2010 at 7:53 PM, Maciej Stachowiak m...@apple.com wrote: On Mar 1, 2010, at 6:46 PM, Ojan Vafai wrote: On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak m...@apple.com wrote: On the one hand, it's good to

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-09 Thread Robert O'Callahan
Mozilla has been using this technique for years. Perhaps we can pick their brains for some good tricks. Or, dare I say it, share some tests. Hi, hope I'm not crashing the party, and sorry I'm late :-). Let me just say a few things about reftests... Maciej mentioned that a reftest can assert

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-09 Thread Jeremy Orlow
On Tue, Mar 9, 2010 at 9:19 AM, Robert O'Callahan rob...@ocallahan.orgwrote: Mozilla has been using this technique for years. Perhaps we can pick their brains for some good tricks. Or, dare I say it, share some tests. Hi, hope I'm not crashing the party, and sorry I'm late :-). Let me just

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-01 Thread Ojan Vafai
On Tue, Feb 23, 2010 at 3:26 PM, Eric Seidel e...@webkit.org wrote: On Tue, Feb 23, 2010 at 2:19 PM, Darin Adler da...@apple.com wrote: But in practice pixel results are often ignored entirely. I think that reftest-style tests if done right could be a great addition. Also, there are

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-01 Thread Maciej Stachowiak
On Mar 1, 2010, at 5:49 PM, Ojan Vafai wrote: On Tue, Feb 23, 2010 at 3:26 PM, Eric Seidel e...@webkit.org wrote: On Tue, Feb 23, 2010 at 2:19 PM, Darin Adler da...@apple.com wrote: But in practice pixel results are often ignored entirely. I think that reftest-style tests if done right

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-01 Thread Ojan Vafai
On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak m...@apple.com wrote: On Mar 1, 2010, at 5:49 PM, Ojan Vafai wrote: On Tue, Feb 23, 2010 at 3:26 PM, Eric Seidel e...@webkit.org wrote: On Tue, Feb 23, 2010 at 2:19 PM, Darin Adler da...@apple.com wrote: But in practice pixel results are

[webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Tor Arne Vestbø
Hey all, A reoccurring problem when trying to maintain layout-test results is differences in font and theme metrics for tests that dump the render tree. Often a test does not actually test font loading/rendering or theming, but has a piece of text or an input element somewhere in the test

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Evan Martin
On Tue, Feb 23, 2010 at 2:00 PM, Tor Arne Vestbø tor.arne.ves...@nokia.com wrote: Lately we've been playing with the idea of using SVG fonts for the Qt port to get the same set of expected results for qt-mac, qt-linux and qt-win, by injecting new @font-face rules using a user-stylesheet and

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Tor Arne Vestbø
On 23/2/10 14:15 , Evan Martin wrote: On Tue, Feb 23, 2010 at 2:00 PM, Tor Arne Vestbø tor.arne.ves...@nokia.com wrote: Lately we've been playing with the idea of using SVG fonts for the Qt port to get the same set of expected results for qt-mac, qt-linux and qt-win, by injecting new

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Simon Fraser
On Feb 23, 2010, at 5:00 AM, Tor Arne Vestbø wrote: Hey all, A reoccurring problem when trying to maintain layout-test results is differences in font and theme metrics for tests that dump the render tree. Often a test does not actually test font loading/rendering or theming, but has a

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Ariya Hidayat
When the test is run, the browser loads both files, takes snapshots, and does a pixel comparison. Thus font differences between platforms become less of an issue. Isn't it our existing pixel test infrastructure? Or do you mean something else? -- Ariya Hidayat

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Darin Adler
On Feb 23, 2010, at 8:08 AM, Ariya Hidayat wrote: When the test is run, the browser loads both files, takes snapshots, and does a pixel comparison. Thus font differences between platforms become less of an issue. Isn't it our existing pixel test infrastructure? Or do you mean something

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Tor Arne Vestbø
On 23/2/10 17:02 , Simon Fraser wrote: On Feb 23, 2010, at 5:00 AM, Tor Arne Vestbø wrote: Hey all, A reoccurring problem when trying to maintain layout-test results is differences in font and theme metrics for tests that dump the render tree. Often a test does not actually test font

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Simon Fraser
On Feb 23, 2010, at 8:21 AM, Tor Arne Vestbø wrote: On 23/2/10 17:02 , Simon Fraser wrote: On Feb 23, 2010, at 5:00 AM, Tor Arne Vestbø wrote: Hey all, A reoccurring problem when trying to maintain layout-test results is differences in font and theme metrics for tests that dump the

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Tor Arne Vestbø
On 23/2/10 17:34 , Simon Fraser wrote: On Feb 23, 2010, at 8:21 AM, Tor Arne Vestbø wrote: On 23/2/10 17:02 , Simon Fraser wrote: I think the correct longterm solution to this problem is to use reftests. A reftest consists of two files; the test file, and a reference file that should give

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Dirk Schulze
Am Dienstag, den 23.02.2010, 08:34 -0800 schrieb Simon Fraser: It could be an image, or it could be a configuration of div elements, or a table, or something else that can be configured to look exactly the same as the CSS border property being tested. Simon I like the idea of reftests.

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Darin Adler
On Feb 23, 2010, at 9:58 AM, Dirk Schulze wrote: But how do we write tests for box-shadows, gradients or patterns on a text (like in SVG)? Possibly with canvas. -- Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Sam Weinig
On Tue, Feb 23, 2010 at 12:58 PM, Dirk Schulze vb...@gmx.de wrote: Am Dienstag, den 23.02.2010, 08:34 -0800 schrieb Simon Fraser: It could be an image, or it could be a configuration of div elements, or a table, or something else that can be configured to look exactly the same as the CSS

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Andras Becsi
Hi, I like this concept of reftests because it seems easier to maintain and scales better on multiple ports than the current form of testing. On the other hand it has to be noted that it would apparently be much work to create reference tests for the current tests which doesn't appear to be a

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread David Levin
On Tue, Feb 23, 2010 at 11:06 AM, Andras Becsi abe...@inf.u-szeged.huwrote: BTW, does the python based layout testing framework, which was checked in recently, support pixel tests? AFAIK it can run multiple DRT's in parallel and speed up the testing process which wouldn't be bad a bad thing

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Dimitri Glazkov
I agree. We should organize cross-browser-test-fest and pick each other's brains. I like the idea of introducing ref tests and I think they are teh hot. My only concern would be to the amount effort of converting existing tests to that, so I think we should start small. :DG On Tue, Feb 23, 2010

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Darin Fisher
The first step is probably to get the infrastructure in place so that new tests can be written as ref tests when possible. -Darin On Tue, Feb 23, 2010 at 11:24 AM, Dimitri Glazkov dglaz...@chromium.orgwrote: I agree. We should organize cross-browser-test-fest and pick each other's brains.

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Darin Adler
On Feb 23, 2010, at 11:06 AM, Andras Becsi wrote: I like this concept of reftests because it seems easier to maintain and scales better on multiple ports than the current form of testing. A reftest-style test seems like a good alternative to a pure text test. It can handle testing a lot more

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Cameron McCormack
Dimitri Glazkov: I agree. We should organize cross-browser-test-fest and pick each other's brains. I believe some Mozilla people are interested in working on a cross browser test suite framework, based in part on reftests. Jonathan Watt and fantasai are people to coordinate with on this, IIRC.

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Eric Seidel
On Tue, Feb 23, 2010 at 2:19 PM, Darin Adler da...@apple.com wrote: But in practice pixel results are often ignored entirely. I think that reftest-style tests if done right could be a great addition. Pixel tests are run for every build by chromium, and regressions tracked there. :) Also,

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Maciej Stachowiak
On Feb 23, 2010, at 8:14 AM, Darin Adler wrote: On Feb 23, 2010, at 8:08 AM, Ariya Hidayat wrote: When the test is run, the browser loads both files, takes snapshots, and does a pixel comparison. Thus font differences between platforms become less of an issue. Isn't it our existing