On Thu, Dec 3, 2009 at 10:25 AM, Ojan Vafai <o...@chromium.org> wrote:
> On Thu, Dec 3, 2009 at 10:21 AM, Dimitri Glazkov <dglaz...@google.com>
> wrote:
>
> How about we turn red for unexpected crashiness?
>>
>
> Makes sense to me. We can just not retry tests that unexpectedly crash.
>
> On Thu, Dec 3, 2009 at 10:09 AM, Dirk Pranke <dpra...@google.com> wrote:
>
>> On Thu, Dec 3, 2009 at 10:07 AM, Ojan Vafai <o...@google.com> wrote:
>> > The test is consistently crashing when run with all the other tests, but
>> > passing when we retry it in isolation. Note that the test is listed as
>> an
>> > unexpected flaky test on the waterfall. This is one of the downsides of
>> > retrying failing tests. We can't distinguish flakiness from this case.
>> We
>> > just need to careful to not ignore unexpected flakiness on the
>> waterfall.
>> > Note that the dashboard only shows the result from the first run.
>> Including
>> > the retry results from the bots seems like more trouble than it's worth.
>>
>> Agreed. However, why aren't the webkit bots orange in the main waterfall?
>>
>
> They are orange. We don't show orange in the console view or summary view
> at the top part of the waterfall though.
>
I'm not really sure why. Nicolas, you know?
>
No idea, did you not write this? :)
> I know in the console view we use orange to mean something else, but maybe
> we should not overload the meaning of orange so much. :)
>
I think it's the goal of the console view to try hard not to attribute
flakiness to a change that most likely did not cause it.
The flakiness dashboard does a good job tracking this already.
That said, if someone can find a good color for "fail again", then I'd use
it and use orange like on the waterfall.
Nicolas
--
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
