On 2012.2.10 4:26 AM, David Golden wrote: > On Thu, Feb 9, 2012 at 11:44 PM, Andreas J. Koenig > <andreas.koenig.7os6v...@franz.ak.mind.de> wrote: >> It's often just Test::More 1.005000_002 and I'm tempted to say we've >> seen enough of it. > > I agree. > > I don't think it was appropriate to smoke with a dev release of > Test::More in the first place. In the future, I'd like to see that > sort of smoke done as an offline regression test. Hopefully, someday > we'll collectively find the tuits to make that a lot easier than it is > today.
I strongly disagree. It is absolutely appropriate to smoke alpha releases. If we're not testing alphas then what's the point of an alpha? If you only smoke stables, the damage is already done! The point of testing is to find bugs. This only happens if tests are allowed to FAIL. If tests aren't failing, or if your system cannot gracefully handle failing tests, you're NOT FINDING BUGS defeating the whole point of the system. Test::More 1.5 is an outrider because it has such a broad effect and has been in alpha so long, but the basic problem with CPAN testers is there: alphas break tests by design and people don't like seeing failing tests that aren't their fault. That's a valid problem, but the solution is not to stop testing alphas. The solution is to separate reports which contain alpha dependencies. Change how they look, change how they're reported, change it so they can be quickly distinguished from a normal failure. We already have the ability to separate alpha and patched versions of Perl. Maybe report it to BOTH parties, the module being tested and the alpha dependency, then the dependent author isn't solely responsible for informing the dependency. You can remove Test::More 1.5 from the rotation, but it's just the latest symptom. This has been a problem since CPAN reporters started communicating results back to authors. I say leave Test::More 1.5 in the system and use it as the itch for somebody to scratch and truly make CPAN testers welcome alphas. Maybe as a QA Hackathon thing. -- Who invented the eponym?
