On Fri, Aug 31, 2012 at 11:53 AM, Daniel Dunbar <[email protected]> wrote:
> On Fri, Aug 31, 2012 at 11:23 AM, David Blaikie <[email protected]> > wrote: > > Daniel (& anyone else), > > > > Do you know if the documentation here ( > > http://llvm.org/docs/TestingGuide.html#rtcustom / > > > http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/TestingGuide.html?view=diff&r1=36059&r2=36060 > > ) is still valid? It doesn't appear to be, since my change below > > doesn't do particularly interesting quoting/escaping gymnastics & > > seems to work fine. > > > > Any ideas? Should we update the documentation? Does anyone know > > what/when/how/why this changed? > > What is actually going on here is that the LLVM test suite and the > Clang test suite use a different style of test. > The LLVM test suite is technically written using Tcl syntax, and the > Clang test suite uses shell syntax. The document in question is > referring to the LLVM test suite. > Is this still true? I tried to switch the LLVM test suite to use the ShTest syntax. I thought I had succeeded. I've got a TODO to go and delete all the lit code for supporting the TclTest variant. > > The fact that the test suites use different styles is an unfortunate > and poorly documented historical accident. Ideally we would kill off > the Tcl style and only use the shell style. > > Also, what is a "lesser lit-like" test runner? > > - Daniel > > > > > Thanks, > > - David > > > > On Fri, Aug 31, 2012 at 10:49 AM, David Blaikie <[email protected]> > wrote: > >> Author: dblaikie > >> Date: Fri Aug 31 12:49:33 2012 > >> New Revision: 163009 > >> > >> URL: http://llvm.org/viewvc/llvm-project?rev=163009&view=rev > >> Log: > >> Fix tests to be more robust (to older versions of grep, lesser lit-like > test runners, etc) > >> > >> Seems I had a problem with my version of grep, when run by lit, not > supporting > >> the \s escape. This seems to fix it for me & I'll be getting the > buildbots to > >> run these tests too to keep an eye on them (actually loop-convert tests > still > >> fail when run via a make build, so that'll be addressed in a future > commit). I > >> could use [[:space:]] to generalize over other whitespace but that > seemed > >> unnecessarily verbose when the flexibility wasn't actually required by > the > >> current text of the tests. > >> > >> Also I just simplified a lot of the loop-convert tests (removing the > >> unecessary temp file deletion at the start, removing the unnecessary && > for > >> FileCheck, etc). > >> > >> The remove-cstr-calls/basic.cpp changes were necessitated by an out of > tree > >> lit-like test runner that's a bit less fantastic about escaping. They > were > >> modeled on existing tooling test cases in Clang, with thanks to Manuel > Klimek > >> for the pointers. > >> > >> Modified: > >> clang-tools-extra/trunk/test/loop-convert/array.cpp > >> clang-tools-extra/trunk/test/loop-convert/confidence.cpp > >> clang-tools-extra/trunk/test/loop-convert/dependency.cpp > >> clang-tools-extra/trunk/test/loop-convert/iterator.cpp > >> clang-tools-extra/trunk/test/loop-convert/naming.cpp > >> clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp > >> > clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp > >> > clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp > >> clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp > >> clang-tools-extra/trunk/test/loop-convert/negative.cpp > >> clang-tools-extra/trunk/test/loop-convert/nesting.cpp > >> clang-tools-extra/trunk/test/loop-convert/nocompile.cpp > >> clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp > >> clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp > >> clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp > >> > >> Modified: clang-tools-extra/trunk/test/loop-convert/array.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/array.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/array.cpp (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/array.cpp Fri Aug 31 > 12:49:33 2012 > >> @@ -1,12 +1,11 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> // RUN: cp %t.cpp %t.base > >> -// RUN: loop-convert -count-only . %t.cpp -- -I %S/Inputs > %T/out \ > >> -// RUN: && FileCheck -check-prefix=COUNTONLY > -input-file=%T/out %s \ > >> -// RUN: && diff %t.cpp %t.base > >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> +// RUN: cp %t.base %t.cpp > >> +// RUN: loop-convert -count-only . %t.cpp -- -I %S/Inputs > %T/out > >> +// RUN: FileCheck -check-prefix=COUNTONLY -input-file=%T/out %s > >> +// RUN: diff %t.cpp %t.base > >> > >> #include "structures.h" > >> > >> > >> Modified: clang-tools-extra/trunk/test/loop-convert/confidence.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/confidence.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/confidence.cpp (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/confidence.cpp Fri Aug 31 > 12:49:33 2012 > >> @@ -1,9 +1,8 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> -// RUN: loop-convert . %t.cpp -A2 -- -I %S/Inputs \ > >> -// RUN: && FileCheck -check-prefix=RISKY -input-file=%t.cpp %s > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> +// RUN: loop-convert . %t.cpp -A2 -- -I %S/Inputs > >> +// RUN: FileCheck -check-prefix=RISKY -input-file=%t.cpp %s > >> > >> #include "structures.h" > >> > >> > >> Modified: clang-tools-extra/trunk/test/loop-convert/dependency.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/dependency.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/dependency.cpp (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/dependency.cpp Fri Aug 31 > 12:49:33 2012 > >> @@ -1,5 +1,4 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> // RUN: loop-convert . %t.cpp -- && FileCheck -input-file=%t.cpp %s > >> > >> void f() { > >> > >> Modified: clang-tools-extra/trunk/test/loop-convert/iterator.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/iterator.cpp (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/iterator.cpp Fri Aug 31 > 12:49:33 2012 > >> @@ -1,9 +1,6 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: rm -rf %t.cpp > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> > >> #include "structures.h" > >> > >> > >> Modified: clang-tools-extra/trunk/test/loop-convert/naming.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/naming.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/naming.cpp (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/naming.cpp Fri Aug 31 > 12:49:33 2012 > >> @@ -1,7 +1,6 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> > >> #include "structures.h" > >> > >> > >> Modified: > clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp > (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp Fri > Aug 31 12:49:33 2012 > >> @@ -1,7 +1,6 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> > >> #include "structures.h" > >> > >> > >> Modified: > clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- > clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp > (original) > >> +++ > clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp Fri > Aug 31 12:49:33 2012 > >> @@ -1,7 +1,6 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert -A0 . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: loop-convert -A0 . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> > >> #include "structures.h" > >> > >> > >> Modified: > clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- > clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp > (original) > >> +++ > clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp > Fri Aug 31 12:49:33 2012 > >> @@ -1,7 +1,6 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> > >> #include "structures.h" > >> > >> > >> Modified: > clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp > (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp > Fri Aug 31 12:49:33 2012 > >> @@ -1,7 +1,6 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> > >> #include "structures.h" > >> > >> > >> Modified: clang-tools-extra/trunk/test/loop-convert/negative.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/negative.cpp (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/negative.cpp Fri Aug 31 > 12:49:33 2012 > >> @@ -1,11 +1,9 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %S/Inputs/negative-header.h > \ > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: grep -Ev "// *[A-Z-]+:" %S/Inputs/negative-header.h > \ > >> // RUN: %T/negative-header.h > >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs/ \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s \ > >> -// RUN: && FileCheck -input-file=%T/negative-header.h \ > >> -// RUN: %S/Inputs/negative-header.h > >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs/ > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> +// RUN: FileCheck -input-file=%T/negative-header.h > %S/Inputs/negative-header.h > >> > >> #include "negative-header.h" > >> #include "structures.h" > >> > >> Modified: clang-tools-extra/trunk/test/loop-convert/nesting.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/nesting.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/nesting.cpp (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/nesting.cpp Fri Aug 31 > 12:49:33 2012 > >> @@ -1,7 +1,6 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> > >> #include "structures.h" > >> > >> > >> Modified: clang-tools-extra/trunk/test/loop-convert/nocompile.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/nocompile.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/nocompile.cpp (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/nocompile.cpp Fri Aug 31 > 12:49:33 2012 > >> @@ -1,8 +1,6 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ > >> -// RUN: || FileCheck -input-file=%t.cpp %s > >> -// Note that this test expects the compilation to fail! > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: not loop-convert . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> > >> void valid() { > >> const int arr[5]; > >> > >> Modified: clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp Fri Aug > 31 12:49:33 2012 > >> @@ -1,8 +1,6 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> -// RUN: rm -rf %t.cpp > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> #include "structures.h" > >> > >> const int N = 6; > >> > >> Modified: clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp > (original) > >> +++ clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp Fri > Aug 31 12:49:33 2012 > >> @@ -1,9 +1,6 @@ > >> -// RUN: rm -rf %t.cpp > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \ > >> -// RUN: && FileCheck -input-file=%t.cpp %s > >> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp > >> -// RUN: rm -rf %t.cpp > >> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp > >> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs > >> +// RUN: FileCheck -input-file=%t.cpp %s > >> > >> #include "structures.h" > >> > >> > >> Modified: clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp > >> URL: > http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp?rev=163009&r1=163008&r2=163009&view=diff > >> > ============================================================================== > >> --- clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp (original) > >> +++ clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp Fri Aug 31 > 12:49:33 2012 > >> @@ -1,7 +1,9 @@ > >> -// RUN: echo '[{"directory":".","command":"clang++ -c > %T/test.cpp","file":"%T/test.cpp"}]' > %T/compile_commands.json > >> -// RUN: cp "%s" "%T/test.cpp" > >> -// RUN: remove-cstr-calls "%T" "%T/test.cpp" > >> -// RUN: cat "%T/test.cpp" | FileCheck %s > >> +// RUN: rm -rf %t > >> +// RUN: mkdir %t > >> +// RUN: echo "[{\"directory\":\".\",\"command\":\"clang++ -c > %t/test.cpp\",\"file\":\"%t/test.cpp\"}]" | sed -e 's/\\/\//g' > > %t/compile_commands.json > >> +// RUN: cp "%s" "%t/test.cpp" > >> +// RUN: remove-cstr-calls "%t" "%t/test.cpp" > >> +// RUN: cat "%t/test.cpp" | FileCheck %s > >> // REQUIRES: shell > >> // FIXME: implement a mode for refactoring tools that takes input from > stdin > >> // and writes output to stdout for easier testing of tools. > >> > >> > >> _______________________________________________ > >> cfe-commits mailing list > >> [email protected] > >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits > _______________________________________________ > cfe-commits mailing list > [email protected] > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits >
_______________________________________________ cfe-commits mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
