Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Dirk Pranke
On Wed, May 23, 2012 at 2:59 PM, Jacob Goldstein wrote: > As a side note to this discussion, there is talk in the W3C community > regarding their test approval process.  At the recent working group > meetings in Germany the idea was floated to simply approve all tests that > are currently waiting

Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Maciej Stachowiak
On May 23, 2012, at 3:13 PM, Dirk Pranke wrote: > On Wed, May 23, 2012 at 2:30 PM, Maciej Stachowiak wrote: >> >> Are you concerned just about the actual pixel results or also about keeping >> render tree dumps up to date? > > Both are more maintenance than a text-only test. In my experience

Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Dirk Pranke
On Wed, May 23, 2012 at 2:30 PM, Maciej Stachowiak wrote: > > Are you concerned just about the actual pixel results or also about keeping > render tree dumps up to date? Both are more maintenance than a text-only test. In my experience, maintaining pixel tests is more expensive, but I also don't

Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Jacob Goldstein
On 5/23/12 2:30 PM, "Maciej Stachowiak" wrote: > >On May 23, 2012, at 2:16 PM, Dirk Pranke wrote: > >> On Wed, May 23, 2012 at 1:41 PM, Ryosuke Niwa wrote: > The only sane argument I've heard so far to gate pixel tests is that >the > correctness of such tests need to be manually i

Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Maciej Stachowiak
On May 23, 2012, at 2:16 PM, Dirk Pranke wrote: > On Wed, May 23, 2012 at 1:41 PM, Ryosuke Niwa wrote: The only sane argument I've heard so far to gate pixel tests is that the correctness of such tests need to be manually inspected, which requires a lot of manual labor and i

Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Dirk Pranke
On Wed, May 23, 2012 at 1:41 PM, Ryosuke Niwa wrote: >> > The only sane argument I've heard so far to gate pixel tests is that the >> > correctness of such tests need to be manually inspected, which requires >> > a >> > lot of manual labor and is very error prone. >> >> I'm assuming the above incl

Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Ryosuke Niwa
On Wed, May 23, 2012 at 1:25 PM, Dirk Pranke wrote: > > Not so: if the tests we had had 100% coverage, then importing more > tests would buy us nothing, but getting rid of the existing tests > would be quite unfortunate. We certainly don't have 100% test coverage. Clearly adding tests incurs so

Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Jacob Goldstein
Dirk, my apologies, I was on travel the week you replied and missed your message. I found it and will review / update now. On 5/23/12 1:25 PM, "Dirk Pranke" wrote: >On Wed, May 23, 2012 at 11:56 AM, Ryosuke Niwa wrote: >> As I have said in the past, we should just import all tests, and treat

Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Dirk Pranke
On Wed, May 23, 2012 at 11:56 AM, Ryosuke Niwa wrote: > As I have said in the past, we should just import all tests, and treat > non-text, non-ref tests as pixel tests. If we wanted to reduce the number of > pixel tests we import, then we should submit those patches to W3C instead of > directly su

Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Jacob Goldstein
I agree. If nothing else, getting W3C tests into the WebKit repository will help catch regressions. From: Ryosuke Niwa mailto:rn...@webkit.org>> To: Jacob Goldstein mailto:jac...@adobe.com>> Cc: WebKit Development mailto:webkit-dev@lists.webkit.org>> Subject: Re: [webkit-

Re: [webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Ryosuke Niwa
As I have said in the past, we should just import all tests, and treat non-text, non-ref tests as pixel tests. If we wanted to reduce the number of pixel tests we import, then we should submit those patches to W3C instead of directly submitting them to WebKit. In general, I don't buy the argument

[webkit-dev] Importing W3C tests to webkit

2012-05-23 Thread Jacob Goldstein
At the WebKit contributor's meeting in April, we discussed a process for importing third party tests into the WebKit repository (specifically, from the W3C test repository). I documented the process that we came up with at the meeting on the WebKit wiki, here: http://trac.webkit.org/wiki/Import