On Mar 17, 2010, at 2:31 AM, Kenneth Russell wrote:
Our best current plan is more widespread testing. We will file a Radar
bug as soon as we have more information about the nature of the
failure -- by virtue of working around the bugs. If we knew the
precise hardware configuration of the bots, including graphics cards,
that would help.
Mark Rowe or Stephanie Lewis would probably know the exact
configurations, I will see if one of them can get you the data tomorrow.
Again, I apologize for the breakage. It would be best for
everyone, I
think, if we got the tree to a green state and all committed through
the queue, thereby having a line of defense against unexpected test
failures on the bots.
If we really want everyone to use the commit queue for most normal
work, we
really have to fix it so that it puts a meaningful value in SVN's
committer
field.
That being said, the mechanism I'd really like to see first is better
notification of when the bot goes red (I suspect a number of people
involved
in today's redness didn't notice right away because there is no
active
notification system).
For what it's worth, the tree was red before I committed due to at
least two or three flaky tests which have been showing up for days. An
active notification system would not have helped here.
The SnowLeopard bot went from 3 failures to over 20 with your commit.
The Leopard Debug bot went from 0 failures to 4. I would think those
are the kinds of events that a computer program could detect and report.
(I also suspect Alexey hadn't noticed that his new tests were failing
on the Gtk and Qt bots, and Daniel didn't notice that his was failing
on Windows.)
I think what
should be done is to get the tree green by skipping these flaky tests;
file high-priority bugs against the test authors to fix the flakiness;
and then figure out a way the commit queue can be used for the vast
majority of patches. (A secure repository of committer
username/passwords on the machine actually executing the commit?)
Were the tests flaky because they were badly designed new tests or
because some patch caused them to start flaking out? It does seem like
the SnowLeopard Release bot, at least, has been failing for some time.
I've looked back to Monday morning (which seems to be as far back as
we have data) and could not find two consecutive green builds of
SnowLeopard Release tests. (Leopard bots, on the other hand, seem to
be green going pretty far back, which makes me wonder if there might
be something wrong with the SL build slaves.)
Regards,
Maciej
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
