On Tue, Feb 5, 2013 at 2:08 PM, Daniel Dunbar <[email protected]> wrote: > > > > On Tue, Feb 5, 2013 at 1:07 PM, David Blaikie <[email protected]> wrote: >> >> On Tue, Feb 5, 2013 at 1:03 PM, Daniel Dunbar <[email protected]> wrote: >> > Author: ddunbar >> > Date: Tue Feb 5 15:03:25 2013 >> > New Revision: 174440 >> > >> > URL: http://llvm.org/viewvc/llvm-project?rev=174440&view=rev >> > Log: >> > [tests] Add support for REQUIRES and XFAIL lines in libc++ tests. >> > >> > - We parse up to the first non-empty non-comment (C++ style) line, >> > otherwise >> > the format and semantics match what is used for LLVM/Clang tests. >> >> Any particular reason for the difference? > > > Most tests don't have metadata
I'm not sure which metadata you're referring to which would influence this aspect. (no doubt there is - I'm just not sure what it is you're referring to) > and we don't have a legacy requirement, this saves parsing time. If that's a preference for all tests should we document it somewhere & intend to cleanup the existing test suites for consistency? (it still seems a little subtle that those first few comments are 'special' & I don't know if the saved parsing time would be noticeable (any numbers?), but people probably do this by default anyway so it doesn't seem like a big issue - but the inconsistency could be annoyingly surprising) > > - Daniel > >> >> > - For now, the only interesting thing to test against is a user >> > supplied >> > target_triple test parameter. >> > >> > Modified: >> > libcxx/trunk/test/lit.cfg >> > >> > Modified: libcxx/trunk/test/lit.cfg >> > URL: >> > http://llvm.org/viewvc/llvm-project/libcxx/trunk/test/lit.cfg?rev=174440&r1=174439&r2=174440&view=diff >> > >> > ============================================================================== >> > --- libcxx/trunk/test/lit.cfg (original) >> > +++ libcxx/trunk/test/lit.cfg Tue Feb 5 15:03:25 2013 >> > @@ -11,6 +11,26 @@ import subprocess >> > import errno >> > import time >> > >> > +# FIXME: For now, this is cribbed from lit.TestRunner, to avoid >> > introducing a >> > +# dependency there. What we more ideally would like to do is lift the >> > "xfail" >> > +# and "requires" handling to be a core lit framework feature. >> > +def isExpectedFail(test, xfails): >> > + # Check if any of the xfails match an available feature or the >> > target. >> > + for item in xfails: >> > + # If this is the wildcard, it always fails. >> > + if item == '*': >> > + return True >> > + >> > + # If this is an exact match for one of the features, it fails. >> > + if item in test.config.available_features: >> > + return True >> > + >> > + # If this is a part of the target triple, it fails. >> > + if item in test.suite.config.target_triple: >> > + return True >> > + >> > + return False >> > + >> > class LibcxxTestFormat(lit.formats.FileBasedTest): >> > """ >> > Custom test format handler for use with the test format use by >> > libc++. >> > @@ -55,6 +75,52 @@ class LibcxxTestFormat(lit.formats.FileB >> > time.sleep(0.1) >> > >> > def _execute(self, test, lit_config): >> > + # Extract test metadata from the test file. >> > + xfails = [] >> > + requires = [] >> > + with open(test.getSourcePath()) as f: >> > + for ln in f: >> > + if 'XFAIL:' in ln: >> > + items = ln[ln.index('XFAIL:') + 6:].split(',') >> > + xfails.extend([s.strip() for s in items]) >> > + elif 'REQUIRES:' in ln: >> > + items = ln[ln.index('REQUIRES:') + 9:].split(',') >> > + requires.extend([s.strip() for s in items]) >> > + elif not ln.startswith("//") and ln.strip(): >> > + # Stop at the first non-empty line that is not a >> > C++ >> > + # comment. >> > + break >> > + >> > + # Check that we have the required features. >> > + # >> > + # FIXME: For now, this is cribbed from lit.TestRunner, to avoid >> > + # introducing a dependency there. What we more ideally would >> > like to do >> > + # is lift the "xfail" and "requires" handling to be a core lit >> > framework >> > + # feature. >> > + missing_required_features = [f for f in requires >> > + if f not in >> > test.config.available_features] >> > + if missing_required_features: >> > + return (lit.Test.UNSUPPORTED, >> > + "Test requires the following features: %s" % ( >> > + ', '.join(missing_required_features),)) >> > + >> > + # Determine if this test is an expected failure. >> > + isXFail = isExpectedFail(test, xfails) >> > + >> > + # Evaluate the test. >> > + result, report = self._evaluate_test(test, lit_config) >> > + >> > + # Convert the test result based on whether this is an expected >> > failure. >> > + if isXFail: >> > + if result != lit.Test.FAIL: >> > + report += "\n\nTest was expected to FAIL, but did >> > not.\n" >> > + result = lit.Test.XPASS >> > + else: >> > + result = lit.Test.XFAIL >> > + >> > + return result, report >> > + >> > + def _evaluate_test(self, test, lit_config): >> > name = test.path_in_suite[-1] >> > source_path = test.getSourcePath() >> > source_dir = os.path.dirname(source_path) >> > @@ -210,4 +276,5 @@ config.test_format = LibcxxTestFormat( >> > ld_flags = ['-nodefaultlibs'] + library_paths + ['-lc++'] + >> > libraries, >> > exec_env = exec_env) >> > >> > -config.target_triple = None >> > +config.target_triple = lit.params.get( >> > + 'target_triple', 'unknown-unknown-unknown') >> > >> > >> > _______________________________________________ >> > cfe-commits mailing list >> > [email protected] >> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits > > _______________________________________________ cfe-commits mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
