On Fri, Aug 31, 2012 at 11:23 AM, David Blaikie <[email protected]> wrote: > Daniel (& anyone else), > > Do you know if the documentation here ( > http://llvm.org/docs/TestingGuide.html#rtcustom / > http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/TestingGuide.html?view=diff&r1=36059&r2=36060 > ) is still valid? It doesn't appear to be, since my change below > doesn't do particularly interesting quoting/escaping gymnastics & > seems to work fine. > > Any ideas? Should we update the documentation? Does anyone know > what/when/how/why this changed?
What is actually going on here is that the LLVM test suite and the Clang test suite use a different style of test. The LLVM test suite is technically written using Tcl syntax, and the Clang test suite uses shell syntax. The document in question is referring to the LLVM test suite. The fact that the test suites use different styles is an unfortunate and poorly documented historical accident. Ideally we would kill off the Tcl style and only use the shell style. Also, what is a "lesser lit-like" test runner? - Daniel > > Thanks, > - David > > On Fri, Aug 31, 2012 at 10:49 AM, David Blaikie <[email protected]> wrote: >> Author: dblaikie >> Date: Fri Aug 31 12:49:33 2012 >> New Revision: 163009 >> >> URL: http://llvm.org/viewvc/llvm-project?rev=163009&view=rev >> Log: >> Fix tests to be more robust (to older versions of grep, lesser lit-like test >> runners, etc) >> >> Seems I had a problem with my version of grep, when run by lit, not >> supporting >> the \s escape. This seems to fix it for me & I'll be getting the buildbots to >> run these tests too to keep an eye on them (actually loop-convert tests still >> fail when run via a make build, so that'll be addressed in a future commit). >> I >> could use [[:space:]] to generalize over other whitespace but that seemed >> unnecessarily verbose when the flexibility wasn't actually required by the >> current text of the tests. >> >> Also I just simplified a lot of the loop-convert tests (removing the >> unecessary temp file deletion at the start, removing the unnecessary && for >> FileCheck, etc). >> >> The remove-cstr-calls/basic.cpp changes were necessitated by an out of tree >> lit-like test runner that's a bit less fantastic about escaping. They were >> modeled on existing tooling test cases in Clang, with thanks to Manuel Klimek >> for the pointers. >> >> Modified: >> clang-tools-extra/trunk/test/loop-convert/array.cpp >> clang-tools-extra/trunk/test/loop-convert/confidence.cpp >> clang-tools-extra/trunk/test/loop-convert/dependency.cpp >> clang-tools-extra/trunk/test/loop-convert/iterator.cpp >> clang-tools-extra/trunk/test/loop-convert/naming.cpp >> clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp >> clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp >> clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp >> clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp >> clang-tools-extra/trunk/test/loop-convert/negative.cpp >> clang-tools-extra/trunk/test/loop-convert/nesting.cpp >> clang-tools-extra/trunk/test/loop-convert/nocompile.cpp >> clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp >> clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp >> clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp >> >> Modified: clang-tools-extra/trunk/test/loop-convert/array.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/array.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/array.cpp (original) >> +++ clang-tools-extra/trunk/test/loop-convert/array.cpp Fri Aug 31 12:49:33 >> 2012 >> @@ -1,12 +1,11 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> // RUN: cp %t.cpp %t.base >> -// RUN: loop-convert -count-only . %t.cpp -- -I %S/Inputs > %T/out \ >> -// RUN: && FileCheck -check-prefix=COUNTONLY -input-file=%T/out %s \ >> -// RUN: && diff %t.cpp %t.base >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> +// RUN: cp %t.base %t.cpp >> +// RUN: loop-convert -count-only . %t.cpp -- -I %S/Inputs > %T/out >> +// RUN: FileCheck -check-prefix=COUNTONLY -input-file=%T/out %s >> +// RUN: diff %t.cpp %t.base >> >> #include "structures.h" >> >> >> Modified: clang-tools-extra/trunk/test/loop-convert/confidence.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/confidence.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/confidence.cpp (original) >> +++ clang-tools-extra/trunk/test/loop-convert/confidence.cpp Fri Aug 31 >> 12:49:33 2012 >> @@ -1,9 +1,8 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> -// RUN: loop-convert . %t.cpp -A2 -- -I %S/Inputs \ >> -// RUN: && FileCheck -check-prefix=RISKY -input-file=%t.cpp %s >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> +// RUN: loop-convert . %t.cpp -A2 -- -I %S/Inputs >> +// RUN: FileCheck -check-prefix=RISKY -input-file=%t.cpp %s >> >> #include "structures.h" >> >> >> Modified: clang-tools-extra/trunk/test/loop-convert/dependency.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/dependency.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/dependency.cpp (original) >> +++ clang-tools-extra/trunk/test/loop-convert/dependency.cpp Fri Aug 31 >> 12:49:33 2012 >> @@ -1,5 +1,4 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> // RUN: loop-convert . %t.cpp -- && FileCheck -input-file=%t.cpp %s >> >> void f() { >> >> Modified: clang-tools-extra/trunk/test/loop-convert/iterator.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/iterator.cpp (original) >> +++ clang-tools-extra/trunk/test/loop-convert/iterator.cpp Fri Aug 31 >> 12:49:33 2012 >> @@ -1,9 +1,6 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: rm -rf %t.cpp >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> >> #include "structures.h" >> >> >> Modified: clang-tools-extra/trunk/test/loop-convert/naming.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/naming.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/naming.cpp (original) >> +++ clang-tools-extra/trunk/test/loop-convert/naming.cpp Fri Aug 31 12:49:33 >> 2012 >> @@ -1,7 +1,6 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> >> #include "structures.h" >> >> >> Modified: clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp >> (original) >> +++ clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp Fri Aug >> 31 12:49:33 2012 >> @@ -1,7 +1,6 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> >> #include "structures.h" >> >> >> Modified: >> clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp >> (original) >> +++ clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp >> Fri Aug 31 12:49:33 2012 >> @@ -1,7 +1,6 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert -A0 . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: loop-convert -A0 . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> >> #include "structures.h" >> >> >> Modified: >> clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp >> (original) >> +++ clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp >> Fri Aug 31 12:49:33 2012 >> @@ -1,7 +1,6 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> >> #include "structures.h" >> >> >> Modified: clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp >> (original) >> +++ clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp Fri >> Aug 31 12:49:33 2012 >> @@ -1,7 +1,6 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> >> #include "structures.h" >> >> >> Modified: clang-tools-extra/trunk/test/loop-convert/negative.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/negative.cpp (original) >> +++ clang-tools-extra/trunk/test/loop-convert/negative.cpp Fri Aug 31 >> 12:49:33 2012 >> @@ -1,11 +1,9 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %S/Inputs/negative-header.h > \ >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: grep -Ev "// *[A-Z-]+:" %S/Inputs/negative-header.h > \ >> // RUN: %T/negative-header.h >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs/ \ >> -// RUN: && FileCheck -input-file=%t.cpp %s \ >> -// RUN: && FileCheck -input-file=%T/negative-header.h \ >> -// RUN: %S/Inputs/negative-header.h >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs/ >> +// RUN: FileCheck -input-file=%t.cpp %s >> +// RUN: FileCheck -input-file=%T/negative-header.h >> %S/Inputs/negative-header.h >> >> #include "negative-header.h" >> #include "structures.h" >> >> Modified: clang-tools-extra/trunk/test/loop-convert/nesting.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/nesting.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/nesting.cpp (original) >> +++ clang-tools-extra/trunk/test/loop-convert/nesting.cpp Fri Aug 31 >> 12:49:33 2012 >> @@ -1,7 +1,6 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> >> #include "structures.h" >> >> >> Modified: clang-tools-extra/trunk/test/loop-convert/nocompile.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/nocompile.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/nocompile.cpp (original) >> +++ clang-tools-extra/trunk/test/loop-convert/nocompile.cpp Fri Aug 31 >> 12:49:33 2012 >> @@ -1,8 +1,6 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >> -// RUN: || FileCheck -input-file=%t.cpp %s >> -// Note that this test expects the compilation to fail! >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: not loop-convert . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> >> void valid() { >> const int arr[5]; >> >> Modified: clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp (original) >> +++ clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp Fri Aug 31 >> 12:49:33 2012 >> @@ -1,8 +1,6 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> -// RUN: rm -rf %t.cpp >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> #include "structures.h" >> >> const int N = 6; >> >> Modified: clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp (original) >> +++ clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp Fri Aug 31 >> 12:49:33 2012 >> @@ -1,9 +1,6 @@ >> -// RUN: rm -rf %t.cpp >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ >> -// RUN: && FileCheck -input-file=%t.cpp %s >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp >> -// RUN: rm -rf %t.cpp >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs >> +// RUN: FileCheck -input-file=%t.cpp %s >> >> #include "structures.h" >> >> >> Modified: clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp >> URL: >> http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp?rev=163009&r1=163008&r2=163009&view=diff >> ============================================================================== >> --- clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp (original) >> +++ clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp Fri Aug 31 >> 12:49:33 2012 >> @@ -1,7 +1,9 @@ >> -// RUN: echo '[{"directory":".","command":"clang++ -c >> %T/test.cpp","file":"%T/test.cpp"}]' > %T/compile_commands.json >> -// RUN: cp "%s" "%T/test.cpp" >> -// RUN: remove-cstr-calls "%T" "%T/test.cpp" >> -// RUN: cat "%T/test.cpp" | FileCheck %s >> +// RUN: rm -rf %t >> +// RUN: mkdir %t >> +// RUN: echo "[{\"directory\":\".\",\"command\":\"clang++ -c >> %t/test.cpp\",\"file\":\"%t/test.cpp\"}]" | sed -e 's/\\/\//g' > >> %t/compile_commands.json >> +// RUN: cp "%s" "%t/test.cpp" >> +// RUN: remove-cstr-calls "%t" "%t/test.cpp" >> +// RUN: cat "%t/test.cpp" | FileCheck %s >> // REQUIRES: shell >> // FIXME: implement a mode for refactoring tools that takes input from stdin >> // and writes output to stdout for easier testing of tools. >> >> >> _______________________________________________ >> cfe-commits mailing list >> [email protected] >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits _______________________________________________ cfe-commits mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
