2009/12/13 Rene Schickbauer <rene.schickba...@gmail.com>:
> Hi!
>
> I'm currently writing some tests for my Maplat framework.
>
> Except for really simple tests, having PostgreSQL server and memcached
> installed is quite essential (both cam be started as temporary instances if
> required as long as the binaries are available).
>
> What is the reasonable response if one or both of them are not available
> during "make test"?
>
> *) FAIL the tests?
If you do this you'll just get spammed to bits.
> *) SKIP the tests?
Maybe but see below.
> *) DIAG("Warning") and skip the tests?
Skip comes with a reason. If you want to give more detail then diag's are fine.
> In my case, skipping the tests will probably exclude > 80% of the
> functionality, so what do i do? I probably can't just assume every
> cpantester has postgresql and memcached installed, can i?
It might be good to factor out all of the independent stuff into its
own module(s) if that makes sense, so that gets widely tested
Basically you want to avoid the tests being run on systems where they
are doomed to fail. You can do that either by
- refusing to install (a bad idea, e.g. pgsl may be installed after your module)
- reducing your dependency by making things work with one of the
lighter in-memory or testing-oriented SQL DBMSs (I think there is at
least 1 pure perl one) and then have that as a prereq for the tests
- reducing your dependency by using a mock database module that is set
up just to respond to the test queries
- skipping them on such systems - I've had big arguments along these
lines before, I think that declaring a "pass" having skipped important
tests due to unsatisfied deps is a bad idea. Users expect a pass to
mean a pass and will probably not even noticed skips whizzing past
during an automated install. Ideally tests should only be skipped when
they are irrelevant - e.g. windows only functions on a linux install.
Skipping them for code that _will_ be called but can't be tested right
now is worse than not testing that code at all - the user is left with
a false-confidence in the module.
- a final odd idea - if you can detect that you are running under
cpan tester (not sure if this is possible), you can dynamically add a
dependency to a sacrifical "postgres_installed" module - this module's
tests always fail if postgres is not available. You will cpantesters
spam about it but you can just /dev/null that. For testers that have
postgresql it will pass and install and then your real module will run
its full test suite,
F
> LG
> Rene
>
> --
> #!/usr/bin/perl # 99 bottles Wikipedia Edition
> $c=99;do{print "$c articles of wikipedia on the net,\n$c articles of wiki".
> "pedia,\ndiscuss one at length, delete it at will,\n".--$c." articles of ".
> "wikipedia on the net.\n\n";}while($c);print"So long & thx for the fish\n";
>
