Some thoughts on the below: 1) An incremental step towards the below is getting the EWS bots to run the tests in the first place. That would be useful even if they didn't know how to add new expectations to attachments.
2) Architecturally, this will require some changes to how the EWS works. The EWS is decentralized, which basically means that the different bots operate independently. Do implement LaTER, we'd need the bots to coordinate at the end to decide where in the fallback chain to add the new expectations. 3) I suspect this system will be difficult to build with the current state of redness in the tree. You should think carefully about how you'd like to handle the situation where not all the tests are passing when LaTER processes a patch. In general, I like the approach of being able to land patches that change the expected results of test on platforms that I don't have handy on my desk. Adam On Fri, Feb 25, 2011 at 3:14 PM, Dimitri Glazkov <[email protected]> wrote: > (re-sending to webkit-dev, since this idea was just mentioned on #webkit) > > If wishes were fishes, checking layout tests expectations into WebKit > would work like so: > > * Authors of layout tests never need to submit test expectations with > their patches > > * All new patches are flagged by a Layout Test Expectations Resolver > (LaTER from here on). > > * LaTER is a set of EWS bots (one for each platform). It takes the new > patch, applies/builds/runs tests/generates expectations. > > * Newly generated expectations are compared against existing > expectations. A minimal set of changes for each platform is created. > > * This set of generated expectations is merged with the changes in the > original patch, and attached to bug as a new patch, obsoleting the > original patch. LaTER knows not to chew on this patch again. > > * The rest of the process (review/cq/land) is the same as it was before. > > Thoughts? Comments? More crazy ideas? > > :DG< _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
