I was just looking at the buildbot cycle stats at
http://build.chromium.org/buildbot/waterfall/stats and realized that on many
bots the most frequently failing tests are browser_tests and ui_tests.
Then I checked how early they are run by each bot (the earlier we know about
the failure, the earlier we can react). So, for example Chromium XP runs a
lot of slow page cycler tests before browser_tests, and then another bunch
of page cycler tests. page_cycler tests don't fail so frequently, and when
they fail, it's frequently some evil flakiness. When browser_tests do fail
however, it may indicate something more serious.
A similar thing is with XP Tests: we're running mini_installer_tests (which
take about 2 minutes), and then some other things which rarely fail, then UI
tests (which fail frequently), and browser_tests at the end!
I know that some of these cases are just flaky failures. But by knowing
earlier about a failure, we'd have more time to decide what to do with it.
--
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
