On Saturday 28 July 2007 09:38:14 Adam Kennedy wrote:
> For background on this email, see the following entry in my journal.
>
> http://use.perl.org/~Alias/journal/33893
>
> Lately, I've noticed a series of modules that are cargo-culting the use
> of test_pod and test_pod_coverage in their tests.
>
> Not only are we seeing spurious failures for users, but worse some
> authors are putting Test::Pod and Test::Pod::Coverage as dependencies,
> and in extreme cases, Perl::Critic as well.
>
> Now, one argument is that it's the author's fault for not appropriately
> skipping these tests if the Test::Pod module is not available.
>
> However, in most of these cases the author would not have made these
> errors AT ALL if not for the fact that Kwalitee says they should have
> these tests.
>
> The CPANTS Game just makes this pressure worse, by making it competive.
>
> That said, I do see the value of these tests EXISTING in the
> distribution, as their provide a guarentee that any new maintainers for
> the modules will continue to comply with the principles those tests
> implement.
>
> BUT for most of these tests, it is BAD that the end-user EVER run them.
>
> Thus, I would like to propose the following.
>
> 1. That the running of POD-related tests by end users is considered
> harmful.
>
> 2. That the test_pod and test_pod_coverage tests by modified, such that
> these tests check for the mentioning of $ENV{AUTOMATED_TESTING} in the
> tests scripts, ensuring that the tests not only exist, but exist in a
> form that is otherwise benign for end users.
>
> This change to the tests is targetted, in the sense that people who do
> NOT have the test_pod and test_pod_coverage tests will not be further
> penalized, while the people that are doing cargo-culted damaging test
> scripts are the ones most likely to care about the CPANTS game point loss.
>
> Thoughts? Alternative?
I use Module::Build (don't jerk your knee yet, Adam) so that I can easily
override 'make disttest' to run my author tests. I keep them in t/author/.
Build.PL for Test::MockObject should make it a little clearer:
http://search.cpan.org/src/CHROMATIC/Test-MockObject-1.08/Build.PL
There's no reason that someone who isn't me couldn't write something similar
for MakeMaker.
I prefer this approach because I don't have to remember any environment
variables to run all of my tests, and because my normal workflow before a
release runs the distribution tests.
Let me also suggest #3: revise the appropriate CPANTS tests to add Kwalitee
only if the static analysis tests do *not* run by default.
-- c
