Hi Thomas, This looks like a very neat test framework. I would however like the file name to be printed in addition to line number as the test can cover multiple files: the file with the tests and files with utilities functions, or replacement for standard functions, that the tests use.
Regards, Mattias Andrée On Tue, 10 Aug 2021 23:45:43 +0200 Thomas Oltmann <[email protected]> wrote: > Hi everybody! > > Quite some time ago, at slcon19, someone was really interested in my > very unconventional way of unit testing C code, > and wanted me to write a post about it on the mailing list. I'm a > couple years late, but here it is anyway: > > > I've tried multiple well-known C unit testing frameworks in the past, > but they all suffered from the same issues: > > - They're quite large and complicated because they pack lots of > features for huge projects with huge development teams. > For small projects like mine (or pretty much all suckless projects) > most of the offered features are complete overkill. > > - they force you to organize your tests into a fixed hierarchy of > 'test suite' > 'test case' > 'test' or the like. > This is a bad fit for any project but those of one specific size class. > > - they make debugging really hard, because they tend to manage the > overall control flow themselves, > spawning all sorts of threads and child processes, so that merely > attaching a debugger can be a pain. > > - they love to spam info logs, often making it difficult to see > whether anything went wrong or not. > On a side note, some of them have this absurd notion that failing a > couple tests is not a big deal ?! > > > So I wrote my own library, which I called 'dh_cuts' - "Dynamic > Hierarchy C Unit Testing System". > It is shamefully trivial, but served me well so far. > > The central idea is to replace the fixed hierarchy of tests nested in > suites, as found in other frameworks, > with a naming hierarchy that is completely dynamic at runtime. This is > realized as follows: > When you want your code to enter a new nested level in the hierarchy, > you call dh_push("<the name of the level>"), > and when you want to leave it again, you call dh_pop(). > > An example: > > void test_the_flux_capacitor() { > dh_push("flux capacitor"); > ... > dh_assert(condition_1 == true); > ... > dh_push("turning some knobs"); > ... > dh_assert(condition_2 == true); > ... > dh_pop(); > > dh_pop(); > } > > void run_scifi_testsuite() { > dh_push("sci-fi"); > test_the_flux_capacitor(); > dh_pop(); > } > > int main() { > dh_init(stderr); > run_scifi_testsuite(); > } > > Per default, if all asserts are passed, this program will print nothing. > But suppose the test fails because condition_2 is false. In that case, dh_cuts > will print a trace of the part of the hierarchy where the error occurred: > > └ sci-fi > ․․└ flux capacitor > ․․․․└ turning some knobs > ․․․․․․└ triggered assert in line 013: condition_2 == true ← FAIL > > This system is great in terms of debuggability, because your tests can convey > as much diagnostic information via dh_push() as you see fit. > dh_push() even accepts the same formatted messages as printf(), so you can > insert things like iteration counts into the hierarchy, and they won't clutter > the output because they're only shown for asserts that fail: > > void monte_carlo_test() { > dh_push("monte carlo"); > for (int n = 0; n < 1000000; n++) { > dh_push("iteration %d", n); > /* perform the n-th round of randomized testing */ > ... > dh_pop(); > } > dh_pop(); > } > > > There's a handful of other features to dh_cuts that make it more practical: > > - You can use the macro dh_branch like this: > dh_branch( > do_some_stuff(); > more *stuff = ...; > ... > ) > to sandbox the code in the parentheses, meaning if that code crashes, > then code following the dh_branch() macro should still be able to execute. > > - There's multiple different dh_assert() variations for convenience as > well as dh_throw() to unconditionally fail with a custom error > message. > > - dh_summarize() can be used to print a one-line summary of executed > vs failed checks. > > - If you don't want the output to include fancy Unicode sequences, you > can define > DH_OPTION_ASCII_ONLY as 1 before including dh_cuts.h. > > - The entire thing is just a tiny single header library, so you can > simply copy-paste it if you want, no dependency management neccessary. > > > If you're interested, you can find the code here: > https://github.com/tomolt/dh_cuts/blob/master/dh_cuts.h > As a more complete example, I've also attached some testing code for a > basic hashtable implementation to this post. > Quite frankly, I don't expect anybody else to start using it, but I > thought people here might be interested by the idea. > > Cheers, > Thomas Oltmann
