Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-04-12 Thread Ryosuke Niwa
Correction: the person from Opera who tried to add reference files is Geoffrey Sneddon as Anne pointed out. +Ms2ger, +ehsan from Mozilla, and +jgraham from Opera. Also note that both Ehsan and James have pointed out that my concern that a test and the corresponding ref will match for wrong reason

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-04-12 Thread Ryosuke Niwa
I've done more digging and talked with folks from Mozilla and Opera about this topic. Here's a summary. Daniel Glazman from Opera has tried adding reference files for W3C's: http://lists.w3.org/Archives/Public/public-css-testsuite/2010Sep/0030.html However, some people in the CSS working group op

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-04-12 Thread Ojan Vafai
On Thu, Apr 12, 2012 at 12:50 PM, Robert Hogan wrote: > On Thursday 12 April 2012 00:58:47 Ryosuke Niwa wrote: > > On Wed, Apr 11, 2012 at 4:45 PM, Ojan Vafai wrote: > > > I agree with the sentiment that we should be upstreaming these to the > > > W3C, but I don't see why we would require upstrea

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-04-12 Thread Robert Hogan
On Thursday 12 April 2012 00:58:47 Ryosuke Niwa wrote: > On Wed, Apr 11, 2012 at 4:45 PM, Ojan Vafai wrote: > > I agree with the sentiment that we should be upstreaming these to the > > W3C, but I don't see why we would require upstreaming them first > > instead of committing them locally and then

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-04-11 Thread Ryosuke Niwa
On Wed, Apr 11, 2012 at 4:45 PM, Ojan Vafai wrote: > > I agree with the sentiment that we should be upstreaming these to the W3C, > but I don't see why we would require upstreaming them first instead of > committing them locally and then upstreaming them. > How do we know whether a reference file

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-04-11 Thread Ojan Vafai
On Wed, Apr 11, 2012 at 3:48 PM, Dirk Pranke wrote: > On Wed, Apr 11, 2012 at 3:46 PM, Dirk Pranke wrote: > > On Fri, Mar 9, 2012 at 2:49 PM, Sam Weinig wrote: > >> > >> On Mar 7, 2012, at 4:41 PM, Ojan Vafai wrote: > >> > >> I just did a first pass a greening the Chromium Lion > >> bot: http:/

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-04-11 Thread Dirk Pranke
On Wed, Apr 11, 2012 at 3:46 PM, Dirk Pranke wrote: > On Fri, Mar 9, 2012 at 2:49 PM, Sam Weinig wrote: >> >> On Mar 7, 2012, at 4:41 PM, Ojan Vafai wrote: >> >> I just did a first pass a greening the Chromium Lion >> bot: http://trac.webkit.org/changeset/110096. Of these hundreds of tests, >>  ~

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-04-11 Thread Dirk Pranke
On Fri, Mar 9, 2012 at 2:49 PM, Sam Weinig wrote: > > On Mar 7, 2012, at 4:41 PM, Ojan Vafai wrote: > > I just did a first pass a greening the Chromium Lion > bot: http://trac.webkit.org/changeset/110096. Of these hundreds of tests, >  ~99% of them are perfect candidates for being reftests (e.g. t

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-03-09 Thread Sam Weinig
On Mar 7, 2012, at 4:41 PM, Ojan Vafai wrote: > I just did a first pass a greening the Chromium Lion bot: > http://trac.webkit.org/changeset/110096. Of these hundreds of tests, ~99% of > them are perfect candidates for being reftests (e.g. they contain one line of > text and a solid box or tw

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-03-08 Thread Tor Arne Vestbø
On 08.03.12 01:57, Levi Weintraub wrote: On Wed, Mar 7, 2012 at 4:50 PM, Ryosuke Niwa mailto:rn...@webkit.org>> wrote: On Wed, Mar 7, 2012 at 4:44 PM, Darin Fisher mailto:da...@chromium.org>> wrote: Hrm, if the test expectations are customized already for different ports of

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-03-07 Thread Maciej Stachowiak
I too am mildly concerned about references not being sufficiently independent of the tests, which is why I hoped we could get the WG in the business of reviewing references along with tests. However, another possibility is looking at what Mozilla uses for reference for these tests, since those

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-03-07 Thread Ojan Vafai
On Wed, Mar 7, 2012 at 4:50 PM, Ryosuke Niwa wrote: > On Wed, Mar 7, 2012 at 4:44 PM, Darin Fisher wrote: > >> Hrm, if the test expectations are customized already for different ports >> of WebKit, then why not support replacing a PNG file with a HTML file that >> is intended to generate exactly

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-03-07 Thread Levi Weintraub
On Wed, Mar 7, 2012 at 4:50 PM, Ryosuke Niwa wrote: > On Wed, Mar 7, 2012 at 4:44 PM, Darin Fisher wrote: > >> Hrm, if the test expectations are customized already for different ports >> of WebKit, then why not support replacing a PNG file with a HTML file that >> is intended to generate exactly

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-03-07 Thread Ryosuke Niwa
On Wed, Mar 7, 2012 at 4:44 PM, Darin Fisher wrote: > Hrm, if the test expectations are customized already for different ports > of WebKit, then why not support replacing a PNG file with a HTML file that > is intended to generate exactly the same result? How does this impair our > ability to upd

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-03-07 Thread Ojan Vafai
That hadn't occurred to me. You're right, we wouldn't actually modify the test itself. We would just replace the -expected.txt/png with a -expected.html file. Maciej, does that change your opinion on this? On Wed, Mar 7, 2012 at 4:44 PM, Darin Fisher wrote: > Hrm, if the test expectations are cu

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-03-07 Thread Darin Fisher
Hrm, if the test expectations are customized already for different ports of WebKit, then why not support replacing a PNG file with a HTML file that is intended to generate exactly the same result? How does this impair our ability to update the tests? (I realize that our current reftest system may

Re: [webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-03-07 Thread Maciej Stachowiak
I'd prefer we not modify imported test suites. That will just make it more confusing to update. Perhaps future CSS test suites will be changed to a reftest model. Regards, Maciej On Mar 7, 2012, at 1:41 PM, Ojan Vafai wrote: > I just did a first pass a greening the Chromium Lion bot: > http:

[webkit-dev] Is converting pixel tests to reftests kosher for imported libraries?

2012-03-07 Thread Ojan Vafai
I just did a first pass a greening the Chromium Lion bot: http://trac.webkit.org/changeset/110096. Of these hundreds of tests, ~99% of them are perfect candidates for being reftests (e.g. they contain one line of text and a solid box or two under the text), but most of them are in the CSS imported