Yes, possibly. The tool is written explicitly for the Chromium builders' output, so it would be plausible but non-trivial to generalize it. It may not be as useful since I don't think they have a concept of an 'upstream' baseline. Those of you who do a lot of work upstream, would it be worth upstreaming this? Glenn
On Thu, Oct 15, 2009 at 12:44 PM, Marc-Antoine Ruel <[email protected]>wrote: > Actually, can you package this so it could be upstreamed to webkit.org? > > > On Thu, Oct 15, 2009 at 1:09 PM, Glenn Wilson <[email protected]>wrote: > >> The tool lives in >> src/webkit/tools/layout_tests/test_output_formatter{.bat,.sh}, and it can >> actually be run against any builder, not just the canaries. Adding better >> documentation is on the TODO list :) >> >> (Also, re: adding link to waterfall, I think we can do this -- I'm >> investigating now.) >> >> >> On Thu, Oct 15, 2009 at 10:05 AM, Yaar Schnitman <[email protected]>wrote: >> >>> Glenn, this tool is amazing! It really helped me yesterday on my 1st >>> gardening shift. >>> Re: "upstream baselines without running the tool manually" >>> How do I do that? >>> >>> On Thu, Oct 15, 2009 at 10:01 AM, Jeremy Orlow <[email protected]>wrote: >>> >>>> Christmas came early for anyone working on WebKit! Thanks Glen. :-) >>>> >>>> >>>> On Thu, Oct 15, 2009 at 9:17 AM, Erik Corry <[email protected]>wrote: >>>> >>>>> >>>>> This makes me very happy! >>>>> >>>>> 2009/10/15 Glenn Wilson <[email protected]>: >>>>> > Hi Webkit gardeners, >>>>> > The Windows canary bot now generates formatted layout test results on >>>>> each >>>>> > build that had unexpected failing layout tests, which means you can >>>>> quickly >>>>> > see layout test failures, diffs, and upstream baselines without >>>>> running the >>>>> > tool manually. The goal of this output is to help gardeners assess >>>>> what may >>>>> > have changed, and how to best handle the failure. >>>>> > For example, here are the failures for build 13359: >>>>> > >>>>> http://build.chromium.org/buildbot/layout_test_results/webkit-rel-webkit-org/29119/index-13359.html >>>>> > To see the formatted layout test failures for a specific canary >>>>> build, just >>>>> > click on the "layout test results" link in the "archive webkit test >>>>> results" >>>>> > step, then click on the "index-####.html" file. >>>>> > There's still more work to do, including getting the formatter >>>>> working >>>>> > properly on the Mac & Linux canaries, adding flakiness data, adding a >>>>> link >>>>> > directly to the waterfall display, and more. Many thanks go to Ojan, >>>>> > Nicolas and all others who endured my many code reviews, and Eric >>>>> Roman and >>>>> > Dimitri for the original spec and design. >>>>> > Please let me know what you think, and what changes you'd like to >>>>> see. >>>>> > Glenn >>>>> > >>>>> > >>>>> > > >>>>> > >>>>> >>>>> >>>>> >>>>> -- >>>>> Erik Corry, Software Engineer >>>>> Google Denmark ApS. CVR nr. 28 86 69 84 >>>>> c/o Philip & Partners, 7 Vognmagergade, P.O. Box 2227, DK-1018 >>>>> Copenhagen K, Denmark. >>>>> >>>>> >>>>> >>>> >>>> >>>> >>> >> >> >> >> > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
