On Sat, Aug 22, 2009 at 4:29 PM, Jeremy Orlow <jor...@chromium.org> wrote:
> On Sat, Aug 22, 2009 at 4:00 PM, Peter Kasting <pkast...@chromium.org>wrote:
>
>> If you somehow managed to not see any comments in this file, I think
>> you're an outlier.
>>
>
> I was talking about the rebaselining teams, not the act of actually
> rebaselining. If someone's rebaselining a test, then it means we now
> believe it's passing. At that point, the notes are not very interesting,
> right? Are you saying that you looked at all the tests' notes before you
> looked through the results to determine if they should be rebaselined?
>
I certainly looked at them during the process of determining what was going
on, and left several notes of my own.
I don't think I understand your objection. Are you saying notes are useless
or that they're harmful? I don't think either is true. If you're trying to
determine how to fix a layout test, the notes in the file are one of the
first things you see, because you're looking in the file to find the bug #,
what OSes are affected, etc. At that point notes that say what to look for
are useful. If you're trying to determine whether to rebaseline a test,
notes are at worst harmless and at best useful in pointing out some subtlety
that you overlooked if you'd already made your decision. You HAVE to see
the notes because you HAVE to edit the file.
Notes in test_expectations.txt are like comments in source code: A great
boon.
There are different reasons for failing. A layout test could be failing
>>> because of a known bug and then start failing in a different way (later) due
>>> to a regression. When a bug fails in a new way, it's worth taking a quick
>>> look, I think.
>>>
>>
>> Why? Unless the earlier failure has been fixed we can't rebaseline the
>> test. (I ran into a number of tests like this when doing my rebaselining
>> pass.) What is the point of looking again?
>>
>
> In case the new failure is more serious than the earlier one.
>
The only possible reason I could think that would matter is if we're using
this as a source of triage input into which bugs we should fix first. But
we have so many thousands of bugs, nearly all likely to be higher priority
than a second failure in a test we already haven't prioritized fixing, that
I don't consider this a valuable signal.
PK
--~--~---------~--~----~------------~-------~--~----~
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
-~----------~----~----~----~------~----~------~--~---
