Hi all,
There was a talk about using gtest or something else for unit testing. I had
suggested Catch instead, however, I think everyone without looking or trying
anything out said gtest. So to help get a real test written I have various PR
coming with unit tests. I had my interns write tests in both gtest and catch
and report out on what they felt was best and why. The result came back and was
a clear win to using Catch, not gtest.
So what are the pro and cons I was given?
Both support:
- using classes to share data and help with setup and teardown cases
- both provide a main function for calling all the tests
- both allow to break up test into multiple programs or as a one big one test
- call specific or multiple tests via command line options
Differences:
- has larger documentation, however, the design of catch requires less, and
most information needed for a test can be found cleaning in the header
- Catch is only a single headers, much easier to compile with. gtest
requires compiling a support library. the basic setup here is that gtest is
more complex to get the basics working
- The assertion logic is different.
- The difference is in that gtest has a form of
- ASSERT_<TYPE>.. ie ASSERT_EQ, ASSERT_NE, etc...
- Catch as a decoupled form of basic concepts with predicates
- REQUIRE_THAT( result, PRED(VALUE)) such as REQUIRE_THAT( myStr,
EndWith("as a service"))
- Catch allows an easy combination of different predicted or existing with
custom predicates, gtest was not so clear. There was difficulty finding the
right macro to call for certain cases. It was not always obvious, and there are
a LOT of them.
- This is where the documentation for gtest was nice to help find what of
the too many macros to use
- Catch on the other have was a simple look up in a header for something
like equal, or greater, or contains and just use that with the high level
condition statement. This was found to be easier than googling or searching the
massive gtest document.
- Catch has some high level BDD style control flow. The interns did not use
this, but thought if would be useful if I made them write more tests. given
they had to implement some more scenario based tests
- Catch appears easier to extend with custom behaviors, which gtest appears
more difficult. Gtests seems to try to include all the combination defined as a
huge set of macro, etc, which Catch define a means to compose different
combination and tries to provide the basic items to compose what you need, and
if you ned something special, you just add a function and you are done.
- the basic reporting of error was viewed as more useful for Catch.. for
example:
gtest output:
- Running main() from gtest_main.cc[==========] Running 1 test from 1 test
case.[----------] Global test environment set-up.[----------] 1 test from
testgogogo[ RUN ] testgogogo.firstgtest.cc:12: Failure Expected: 1To
be equal to: gogogo(2) Which is: 2[ FAILED ] testgogogo.first (0
ms)[----------] 1 test from testgogogo (0 ms total)
[----------] Global test environment tear-down[==========] 1 test from 1 test
case ran. (0 ms total)[ PASSED ] 0 tests.[ FAILED ] 1 test, listed below:[
FAILED ] testgogogo.first
Catch output:
-
-------------------------------------------------------------------------------test
case
1-------------------------------------------------------------------------------test.cc:17...............................................................................
test.cc:18: FAILED: REQUIRE( gogogo(2) == 1 )with expansion: 2 == 1
===============================================================================test
cases: 1 | 1 failedassertions: 1 | 1 failed
either way That is the main data I have
The summary is recommend we use Catch not gtest for unit testing.I plan to add
to a PR some tests with Catch under tests/unit-tests/
Jason
