Dirk, my apologies, I was on travel the week you replied and missed your message. I found it and will review / update now.
On 5/23/12 1:25 PM, "Dirk Pranke" <dpra...@chromium.org> wrote: >On Wed, May 23, 2012 at 11:56 AM, Ryosuke Niwa <rn...@webkit.org> wrote: >> As I have said in the past, we should just import all tests, and treat >> non-text, non-ref tests as pixel tests. If we wanted to reduce the >>number of >> pixel tests we import, then we should submit those patches to W3C >>instead of >> directly submitting them to WebKit. >> >> In general, I don't buy the argument that adding more pixel tests incurs >> more cost than the benefit we get from importing the tests. If that >>were the >> case, we can just get rid of the existing pixel tests we have. >> > >Not so: if the tests we had had 100% coverage, then importing more >tests would buy us nothing, but getting rid of the existing tests >would be quite unfortunate. Clearly adding tests incurs some costs and >probably provides some benefit; the question is when does the cost >exceed the benefit? > >As I am not against importing more tests per se, I think this only >makes sense to evaluate on a case-by-case basis. > >> The only sane argument I've heard so far to gate pixel tests is that the >> correctness of such tests need to be manually inspected, which requires >>a >> lot of manual labor and is very error prone. >> > >I'm assuming the above includes the ongoing maintenance cost of >keeping pixel tests up to date, as well as the cost at the initial >checkin. > >There is also the fact that the more tests we have, the more tests we >have to run, and increasing cycle time by itself is a cost to >developer productivity. Of course, it's also potentially the case that >we have to update tests from time to time, although this doesn't >happen often. > >Jacob, I gave you a bunch of feedback not long after you published the >initial writeup, but it doesn't look like any of that has been >incorporated? > >-- Dirk _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
