On Saturday 28 July 2007 09:38:14 Adam Kennedy wrote: > For background on this email, see the following entry in my journal. > > http://use.perl.org/~Alias/journal/33893 > > Lately, I've noticed a series of modules that are cargo-culting the use > of test_pod and test_pod_coverage in their tests. > > Not only are we seeing spurious failures for users, but worse some > authors are putting Test::Pod and Test::Pod::Coverage as dependencies, > and in extreme cases, Perl::Critic as well. > > Now, one argument is that it's the author's fault for not appropriately > skipping these tests if the Test::Pod module is not available. > > However, in most of these cases the author would not have made these > errors AT ALL if not for the fact that Kwalitee says they should have > these tests. > > The CPANTS Game just makes this pressure worse, by making it competive. > > That said, I do see the value of these tests EXISTING in the > distribution, as their provide a guarentee that any new maintainers for > the modules will continue to comply with the principles those tests > implement. > > BUT for most of these tests, it is BAD that the end-user EVER run them. > > Thus, I would like to propose the following. > > 1. That the running of POD-related tests by end users is considered > harmful. > > 2. That the test_pod and test_pod_coverage tests by modified, such that > these tests check for the mentioning of $ENV{AUTOMATED_TESTING} in the > tests scripts, ensuring that the tests not only exist, but exist in a > form that is otherwise benign for end users. > > This change to the tests is targetted, in the sense that people who do > NOT have the test_pod and test_pod_coverage tests will not be further > penalized, while the people that are doing cargo-culted damaging test > scripts are the ones most likely to care about the CPANTS game point loss. > > Thoughts? Alternative?
I use Module::Build (don't jerk your knee yet, Adam) so that I can easily override 'make disttest' to run my author tests. I keep them in t/author/. Build.PL for Test::MockObject should make it a little clearer: http://search.cpan.org/src/CHROMATIC/Test-MockObject-1.08/Build.PL There's no reason that someone who isn't me couldn't write something similar for MakeMaker. I prefer this approach because I don't have to remember any environment variables to run all of my tests, and because my normal workflow before a release runs the distribution tests. Let me also suggest #3: revise the appropriate CPANTS tests to add Kwalitee only if the static analysis tests do *not* run by default. -- c