I don't think it's realistic to expect the gardener, or any one person, to
be able to fix an arbitrary broken layout test in a reasonable period of
time. That's certainly true for new tests, but even for regressions I often
can't even tell for sure whether our results are correct, much less what to
do if they're not.
It's far more efficient to have the "right" person fix a test. (Of course,
people should also strive to broaden their knowledge, but there's a limit to
how much of that one can do in a week.) Never having broken layout tests is
an excellent goal, but quite frankly I don't think it's one we should
prioritize so high that we hobble other efforts and burn out developers.
- Pam
On Tue, Oct 13, 2009 at 10:31 AM, Dimitri Glazkov <dglaz...@chromium.org>wrote:
>
> I think we need to change something. I am not sure what -- I have
> ideas, but -- I would appreciate some collective thinking on this.
>
> PROBLEM: We accumulate more test failures via WebKit rolls than we fix
> with our LTTF effort. This ain't right.
>
> ANALYSIS:
>
> Ok, WebKit gardening is hard. So is fixing layout tests. You can't
> call it a successful WebKit roll if it breaks layout tests. But we
> don't revert WebKit rolls. It's a forward-only thing. And we want to
> roll quickly, so that we can react to next "big breaker" faster. So
> we're stuck with roll-now/clean-up-after deal. This sucks, because the
> "clean-up-after" is rarely fully completed. Which brings failing
> layout tests, which brings the suffering and spells asymptotic doom to
> the LTTF effort.
>
> POSSIBLE SOLUTIONS:
>
> * Extend WebKit gardener's duties to 4 days. First two days you roll.
> Next two days you fix layout tests. Not file bugs -- actually fix
> them. The net result of 4 days should be 0 (or less!) new layout test
> failures. This solution kind of expects the gardener to be part of
> LTTF, which is not always the case. So it may not seem totally fair.
>
> * Assign LTTF folks specifically for test clean-up every day. The idea
> here is to slant LTTF effort aggressively toward fixing newer
> failures. This seems nice for the gardeners, but appears to separate
> the action/responsibility dependency: no matter what you roll, the
> LTTF elves will fix it.
>
> * [ your idea goes here ]
>
> TIMELINE:
>
> I would like for us to agree on a solution and make the necessary
> changes to the process today. Tomorrow is a new day, full of
> surprising changes upstream.
>
> :DG<
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
-~----------~----~----~----~------~----~------~--~---
