Hi Alexander,
thanks a lot for the detailed explanations (and the new information
about the sub-projects on the web is great). I finally got around
playing with the deployed bundles and everything worked nicely.
On 06/24/2012 10:02 PM, Alexander Broekhuis wrote:
Is there a reason to not use CTest, which ships with CMake?
I really dislike CTest's behaviour of only having the possibility of one
test in an executable (better said, one result).
I'm not sure if I can follow. In one of our projects we have about 250
tests which are "contained" in about ten separate executables. The CMake
create_test_sourcelist is used to to create a test driver (i.e. a test
suite) which can call (via command line arguments) the individual tests.
add_test() is then used to let CTest know about available tests, e.g. by
calling add_test(NAME ServiceListenerTest COMMAND FrameworkTestSuite
ServiceListenerTest).
An example invocation of CTest where only one "test executable" is used
then looks like this
sascha@BIGEYE ~/builds/CppMicroServices-debug> ctest
Test project /home/sascha/builds/CppMicroServices-debug
Start 1: usDebugOutputTest
1/6 Test #1: usDebugOutputTest ................ Passed 0.00 sec
Start 2: usLDAPFilterTest
2/6 Test #2: usLDAPFilterTest ................. Passed 0.00 sec
Start 3: usModuleTest
3/6 Test #3: usModuleTest ..................... Passed 0.01 sec
Start 4: usServiceListenerTest
4/6 Test #4: usServiceListenerTest ............ Passed 0.01 sec
Start 5: usServiceRegistryTest
5/6 Test #5: usServiceRegistryTest ............ Passed 0.00 sec
Start 6: usServiceTrackerTest
6/6 Test #6: usServiceTrackerTest ............. Passed 0.01 sec
100% tests passed, 0 tests failed out of 6
Total Test time (real) = 0.03 sec
Also since CTest is only a wrapper to starting test, and not a test
framework like CUnit etc, there is still a need for additional libraries
with assertions etc. Since CUnit etc all support test-suites (multiple test
results in one executable), I prefer those over CTest.
In its simplest form, CTest can be viewed just as mechanism to call
executables which do some tests. But it also allows you to selectively
execute tests based on attached "LABEL" properties, or debug/release
specific tests, specify global or individual test timeouts, run tests in
parallel (-j <x>) etc.
You are right that there is no C/C++ support for fancy assert or special
test macros. However, I have found that a small set of custom macros and
a small utility class in the project itself is all I need, e.g. here:
https://github.com/saschazelzer/CppMicroServices/blob/master/test/usTestingMacros.h
https://github.com/saschazelzer/CppMicroServices/blob/master/test/usTestManager.cpp
But there is no "setup" and "teardown" functions, so I guess for more
elaborate testing set-ups an external unit test library makes sense.
Another problem with CTest is reporting. With CTest it is only possible to
create some reports using the VTK "Dashboard". Using some other library
which supports the JUnit XML export makes it easier to integrate the build
with Jenkins/Hudson and other CI tools.
I guess you meant the CDash "Dashboard", which you can install yourself
on any machine to send your CTest reports to. But I fully agree that
integration with other tools is not easy. I have seen a Jenkins CTest
plug-in somewhere, but have no experience with it, and there are some
xslt files in the net for transforming ctest xml to junit xml.
Anyway, I was just used to be able to type "ctest" in the build
directory and expected the tests to trun. Maybe we could add
"add_test()" calls which call the third-party unit test library test driver?
Best,
Sascha
