On 11/09/2011 06:06 PM, EXT-York, Gantry wrote:
> I'm looking for suggestions from someone who has used CTest to report results
> to a dashboard other than CDash.
>
> I'd like to be able to simply type
>
> gmake test TESTOPTS="-report"
>
> then have CTest traverse all the directories and run the defined tests. Not
> only would I like it to run these tests, but I'd like it to run it through a
> wrapper script that I use to report the results to a different dashboard.
>
> I've tried this with
>
> set( TEST_WRAPPER /full/path/to/test_wrapper )
> ...
> add_test( unit_test1 )
> # syntax 1
> add_test( /full/path/to/test_wrapper --report unit_test2 ) # syntax 2
> add_test( ${TEST_WRAPPER} ${TESTOPTS} unit_test3 ) # syntax 3
>
> There are several problems I've noticed with this:
>
> Syntax 1 works fine. It knows where the name unit_test1 is located and runs,
> but since it isn't run through the wrapper script, nothing is reported to our
> dashboard.
Actually, this can't work as it misses the test's name. Moreover, you
should use the more powerful ADD_TEST(NAME ... COMMAND ...) signature.
> Syntax 2 works does not work. All the arguments are quoted. "-report" fails
> as an option because of the quotes, and "unit_test2" is treated as a string
> not as a name of a CMake object with a location.
Use generator expressions to refer to a target as a parameter for a test:
ADD_TEST(... COMMAND path/to/wrapper --report $<TARGET_FILE:unit_test2>)
AFAIK, CMake recognizes a target only as the command's first argument.
> Syntax 3 is just a variation of syntax 2 and also fails, but it is preferable
> to define the wrapper script in the highest level CMakeLists.txt file and
> have that value carried through to sub directories.
Absolutely, and if the script is not contained in or generated by your
project, use FIND_PROGRAM() to locate it and to populate the variable.
> So I tried this
>
> set( TEST_WRAPPER /full/path/to/test_wrapper_report )
> ...
> add_test( ${TEST_WRAPPER_REPORT} "${CURRENT_DIR}/ unit_test4" )
> # syntax 4
>
> I modified the test wrapper to report without having to specify the -report
> option.
>
> In this case, everything seemed to work, but the return/exit value coming
> back from the wrapper script was wrong. It seems that something is going on
> in the CTest framework that changes this value. Instead of getting a value
> like -1, 0, 1, I was getting a value like 32844. I verified that unit_test4
> returned the correct return status to the wrapper script and the wrapper
> script returned the correct return status when run outside of the CTest
> framework.
Usually, it should be possible to specify any arguments to a test
driver as arguments behind ADD_TEST()'s COMMAND clause, and there
should also be no issues w.r.t. quotes. Thus, you might try anew
with generator expressions and correctly set up ADD_TEST()
commands, and report if it still doesn't work.
Regards,
Michael
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
