One more thing (since people are asking), you can see all the platforms' expectations for a test from the view for any builder. The ones that don't apply to this builder are greyed out. For example, see LayoutTests/fast/dom/HTMLObjectElement/object-as-frame.html on the "WebKit" builder. Ojan
On Mon, Aug 24, 2009 at 6:42 PM, Ojan Vafai <[email protected]> wrote: > > A first version of the layout test flakiness dashboard is up. > > http://src.chromium.org/viewvc/chrome/trunk/src/webkit/tools/layout_tests/flakiness_dashboard.html > > Some of the key features: > > - updates roughly as quickly as the bots have run the tests (no > post-processing, stdio parsing or crons) > - sortable by any column include "flakiness" > - builder + sort order permalinks (i.e. you can copy-paste URLs for a > given builder + sort order) > - highlights tests with missing expectations (i.e. the test passed, but > is marked only as failing) > - prints out test run times for tests that took > 1 second > - shows the expectations for a test on all platforms > > Taking a look at the dashboard, you can immediately see a couple things: > > 1. A bunch of cases where we currently have the missing expectations > for a test. For example, LayoutTests/accessibility/plugin.html fails on > Windows, but is only listed as failing on the Mac. > 2. We have a few dozen very slow tests that considerably slow down how > long the tests take to run. > > How do you get this awesomeness for UI tests, unittests, etc? It's actually > *really* easy. Someone just needs to add something to the test runners for > those test types that spits out JSON files in the right format and copies > them to the appropriate place. Let me know if you're interested in adding > support for a new test type. > > Known issues: > > 1. The JS that generates the page could stand to be faster. > 2. The windows builders have a bunch of junk entries with windows style > paths. Ignore them for now, they'll gradually fall off the end of the tests > tracked by the dashboard. Only the tests with unix style paths should be > looked at. > > If you have bugs or feature requests. Please file them at crbug.com and CC > me. One feature request that I have is to also highlight cases where we list > an expectation for a test that never happens (e.g. the test is listed as > "PASS CRASH FAIL", but has never crashed. > > Ojan > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
