On 03/29/2015 11:23 PM, Laurent Bercot wrote:
On 2015-03-28 15:36, Lorenzo wrote:
It's been a month, and no replies... but hey, "please contact me",
does it mean there's been some progress in private discussions?
I wish. But unfortunately, it looks like testing is as unappealing
to other people as it is to me. :)
(Not that I blame anyone. I can't be bothered to write tests for
my own freakin' projects, so why would it be surprising that nobody
wants to write tests for a project they don't even own ? It's really
a case of "there are all kind of fetishes in the world and I'm hoping
to find somebody with a compatible one". Which is a numbers game,
and we don't have the numbers yet.)
If not, I've spent a few hours looking at possible candidates that
match your criteria(*), and some candidates actually survived.
Awesome! I'm all ears.
Please tell me that someone mailed you in private anyway :)
The silence in my mailbox was deafening. The community is still too
small so the odds are against me. :)
I hope to make the community larger with the projects I'm currently
working on; I can still keep things under control until then at least.
A part of me still wishes I never sent that email, you know... ;p
So, I went to check the test suites I sort of knew (and hated), than
those from
https://stackoverflow.com/questions/65820/unit-testing-c-code, then
wikipedia (it's 1e9 projects, which helps explaining why I was
aggressively trying to cut the numbers down).
Oh, and since command line utilities are weird to test, add
https://stackoverflow.com/questions/971945/unit-testing-for-shell-scripts or
something...
Having no extra dependencies cut many of them; having to be included in
the tarball cut most of those left.
That being said, I regret not keeping a log of reasons why I rejected a
project or another - the list is long, it's possible that I just skipped
some of them and I don't know.
Anyway, here's the projects that survived till the end.
TLDR: I vote for tap - who said that nothing good came out of perl?
MINUNIT http://www.jera.com/techinfo/jtns/jtn002.html
-----------------------------------------------------
Should be small and simple enough XD
CUTEST (https://github.com/mity/cutest)
---------------------------------------
Just look at the header and read the examples, it's simplicity itself.
Then look at test_crash(), add another test after it and see what
happens - neat! So neat it rules out seemingly promising projects like
https://github.com/bvdberg/ctest...
Wikipedia says it's used in the APR; this is no longer true since ????,
but at least proves it has real world usage.
The main downside is that it's only for C code, so testing a binary by
running it from C is awkward.
Raspberry?
https://github.com/danfis/cu, https://github.com/imb/fctx,
https://github.com/bvdberg/ctest
---------------------------------------------------------
Examples of projects I discarded because I feared people would think
"ugh, too much to learn - someone else will write the tests anyway".
Especially cu shows that I probably set the bar too high.
TAP (http://testanything.org/)
------------------------------
Remember that text is a universal interface? they do.
TAP is a *very* simple textual protocol to say "test 1 ok, test 2
botched, test 3 ok (extra details)" etc.
My favorite library is http://github.com/zorgnax/libtap - it's <500loc,
looks very sensible, and the author recommends just copying tap.[ch] in
your projects.
+ it's a __protocol__, so there's a lot of documentation; try running
"perldoc Test::Tutorial", the perl api is... well, perl, but the
concepts are very sound, explained clearly and with the assumption of a
reader scared of testing.
+ very sensible: eg it helps you maintain todo lists. Really, look at
Test::Tutorial
- need a program ("consumer") to analyse the output of the testsuite
+ can be very minimal; if tests exit nonzero on failure, just a "for t
in $tests ; do $t||exit; done" is enough to get you started; even the
autotools felt they could write a consumer using the shell...
+ can do fancy stuff, like running tests in parallel etc - see eg the
commands suggested in git's t/README
- I didn't look for a consumer as minimal as libtap
+ MAIN SELLING POINT: not restricted to C, can be used to test
commandline utilities - see the git testsuite, it's *overkill* for us,
but gives you an idea of how well tap can scale if you really need it
+ test gets a SIGSEGV or similar => the consumer notices
+ superlightweight, you can even run the tests and analyze them on
another machine if you want to
- http://www.slideshare.net/fperrad/arduino-tap
- git produces ~500KB of output- a few minutes even with 9600 baud
TODO
Test coverage - aka, how I miss "go test -cover"...
Gcov is gcc only, it that ok? Do you have any platform where it's not
the main compiler?
PS
Sorry that this message is more similar to a braindump than a proper
email, it's almost 1am here and DST started today.
Next time I'll be fully awake!
