I understand the resistance to implement yet another bit of process and effort around layout tests. I really do. However, I found some merit in Dirk's idea -- it allows us to clearly see the impact of a regression.
Sadly, I can't come up with a specific example at the moment, but let me pull one out of my ... hat, based on previous experiences. Let's say we had a regression in JSON parsing. But since we already fail parts of the LayoutTests/fast/js/JSON-parse.html, we wouldn't notice it. Especially with DOM bindings, there are tons of tests like this -- we pass only parts of them, so we wouldn't know when our changes or commits upstream introduce regressions that we really ought to be noticing. It's kind of like marking layout tests as flakey: there's no way to determine whether the flakiness is gone other than by recording some extra information. So to me at least, the benefit of this type of solution is not near-zero. My only hesitation comes from having to decide whether we should stop and implement this rather than dedicate all of our resources to plowing ahead in fixing layout tests and driving the number to 0 (and thus eliminating the need for this solution). :DG< On Fri, Aug 21, 2009 at 6:43 PM, Ojan Vafai<[email protected]> wrote: > On Fri, Aug 21, 2009 at 4:54 PM, Peter Kasting <[email protected]> > wrote: >> >> On Fri, Aug 21, 2009 at 4:50 PM, Dirk Pranke <[email protected]> wrote: >>> >>> This is all good feedback, thanks! To clarify, though: what do you >>> think the cost will be? Perhaps you are assuming things about how I >>> would implement this that are different than what I had in mind. >> >> Some amount of your time, and some amount of space on the bots. > > Also, some amount of the rest of the team's time to follow this process. > Ojan > > > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
