Great work, dude. Seriously good stuff. I'll be digging through this tomorrow.
Now, how do I change the theme on this thing? ;P :DG< On Mon, Aug 24, 2009 at 6:42 PM, Ojan Vafai<o...@chromium.org> wrote: > A first version of the layout test flakiness dashboard is up. > http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/layout_tests/flakiness_dashboard.html > Some of the key features: > > updates roughly as quickly as the bots have run the tests (no > post-processing, stdio parsing or crons) > sortable by any column include "flakiness" > builder + sort order permalinks (i.e. you can copy-paste URLs for a given > builder + sort order) > highlights tests with missing expectations (i.e. the test passed, but is > marked only as failing) > prints out test run times for tests that took > 1 second > shows the expectations for a test on all platforms > > Taking a look at the dashboard, you can immediately see a couple things: > > A bunch of cases where we currently have the missing expectations for a > test. For example, LayoutTests/accessibility/plugin.html fails on Windows, > but is only listed as failing on the Mac. > We have a few dozen very slow tests that considerably slow down how long the > tests take to run. > > How do you get this awesomeness for UI tests, unittests, etc? It's actually > *really* easy. Someone just needs to add something to the test runners for > those test types that spits out JSON files in the right format and copies > them to the appropriate place. Let me know if you're interested in adding > support for a new test type. > Known issues: > > The JS that generates the page could stand to be faster. > The windows builders have a bunch of junk entries with windows style paths. > Ignore them for now, they'll gradually fall off the end of the tests tracked > by the dashboard. Only the tests with unix style paths should be looked at. > > If you have bugs or feature requests. Please file them at crbug.com and CC > me. One feature request that I have is to also highlight cases where we list > an expectation for a test that never happens (e.g. the test is listed as > "PASS CRASH FAIL", but has never crashed. > Ojan > > > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
