Also, for anyone wondering, the "check in failing test baselines" feature will be landing shortly after this sticks. I'll send out another note with details on that when we get there.
-- Dirk On Wed, Sep 12, 2012 at 4:29 PM, Dirk Pranke <dpra...@chromium.org> wrote: > Hi all, > > The new format of the much-debated TestExpectations syntax will be > landing soon (hopefully in the next couple days). > > For those of who have forgotten / repressed the earlier debates, the > new syntax looks something like: > > webkit.org/b/12345 [ Mac Vista] fast/html/keygen.html [ ImageOnlyFailure ] > > Andis documented in full at > https://trac.webkit.org/wiki/TestExpectations#NewSyntaxNotquiteyetlanded > . > > ( The "[" and "]" characters are delimiters, not EBNF "optional" > markers, although those sections are in fact optional :) ). > > Note that the new syntax means that Skipped files are a syntactic > subset of TestExpectations files, and I plan to convert all of the > Skipped files to TestExpectations files via copy and paste shortly > after the new syntax is landed, and then drop support for Skipped > files (I will update ORWT to use the new files and treat any entry as > a Skip). > > The plan for landing these changes is: > > 1) Add support for parsing the new lines and converting them back into > the old format (internally) so that both syntaxes are supported > 2) Convert all the existing files over > 3) Make sure things aren't broken :) > 4) Drop support for the old syntax > > I plan for this to all happen quickly, in less than a day. This means > that if you have patches posted that modify those files they may > become stale and need to be updated. > > Changes from the old syntax: > > 1. We use URLs (a specific whitelisted set; let me know if you want to > add to it) instead of BUGWK12345 etc. > 2. We use "bug(dpranke) instead of BUGDPRANKE > 3. We use CamelCase instead of SHOUTING > 4. We use Failure to represent what used to be TEXT, IMAGE+TEXT, and > AUDIO - these failures will be indistinguishable in the new world, > meaning that you can't distinguish between "text only" and "both image > and text". Since only Chromium runs pixel tests by default, this > shouldn't be a big deal. > 5. We use ImageOnlyFailure to represent what used to be IMAGE > 6. We use "[" and "]" for delimiters instead of ":" and "=" > 7. We use "#" instead of "//" as a comment > 7. WontFix will now imply Skip, i.e., tests marked WontFix will > automatically be Skipped > 8. WontFix and Skip will not require (or even allow) any other > expectations, i.e., you can't say "[ WontFix Crash ]". If you want to > indicate that the test will crash if you actually run it, use a > comment. > 9. WontFix, Skip, Slow, and Rebaseline all move from the left hand > side to the right. The only keywords on the left restrict which > configurations the lines apply to. > > I will send out follow-up emails as this stuff lands. Please let me > know if you have any questions. Thanks! > > -- Dirk _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
