On Mon, Feb 25, 2013 at 12:38 PM, Eric Seidel <[email protected]> wrote: > I've noticed as of late several different approaches being used when > adding/changing LayoutTests which need rebaselining on other platforms. > > Obviously we cannot expect developers to test/rebaseline on all platforms > before landing given our current infrastructure. > > But what should we expect them to do? > > Currently some folks add failing expectations to other ports > TestExpectations. Some add [skip]. Some even add them to the (new) global > TestExpectations file. > > What's the proper course of action? > > I checked: > http://trac.webkit.org/wiki/Keeping%20the%20Tree%20Green > http://trac.webkit.org/wiki/Creating%20and%20Submitting%20Layout%20Tests%20and%20Patches > http://trac.webkit.org/wiki/TestExpectations > and didn't see the "I expect this to fail/need rebaseline on other ports" > case discussed. >
Memory says that the last time this was raised [1], the consensus was to land the change, turn the tree red, notify as many gardeners / port maintainers as possible, and rebaseline as quickly as possible. I.e., don't add entries to TestExpectations. Of course, such a policy wouldn't play nicely w/ the EWS bots, and I think this discussion predated everyone switching over to TestExpectations (and certainly predated the generic expectations file). I don't find the above entirely satisfactory, but I also don't have any great alternatives to endorse given the existing tooling. Also, if the test fails generically (all ports), it probably shouldn't be landed. I'm not sure why you couldn't at least update the port you're developing on; if you don't, how do you know your fix is working at all? (Of course, there could be a case where a high-priority fix might break some low-priority tests). > I remember some discussion of a [rebaseline] keyword in TestExpectations, > but I'm not sure that ever made it in? The [ NeedsRebaseline ] enhancement is, as of yet, unimplemented and unclaimed [2]. It shouldn't be too hard for someone to try it if they're looking for a reason to explore the NRWT code :) -- Dirk [1] https://lists.webkit.org/pipermail/webkit-dev/2012-April/020250.html [2] https://bugs.webkit.org/show_bug.cgi?id=100415 _______________________________________________ webkit-dev mailing list [email protected] https://lists.webkit.org/mailman/listinfo/webkit-dev
