The tenor of the opinions in the passing TODO tests bother me a bit. It seems folks have forgotten why the CPAN installation chain exists. Especially the assertion that its perfectly ok for modules to start failing even though there's nothing wrong with them (unless you reeeeaally stretch things). The install chain is a special place and very conservative, often radically so.
Let's remember that we are not a representative sample of CPAN users. For this reason I state: --->>>*** The primary purpose of the install chain is to install modules ***<<<--- Not to test the module. Not to report statistics back to the author. Not to inform the developer that a TODO test passed. Not for the convenience of CPAN testers. These are all secondary. It has to work. It has to work and it has to work automatically without user intervention unless absolutely necessary. It may not be perfect but that is the goal. I'll paraphrase something Chip said to me at YAPC, "its not that it actually works, but its important that we perceive it not working as a problem". [1] A passing TODO test is not a failure. The code is working better than expected. The scenarios proposed so far where a passing TODO causes trouble are strained, unrealistic and require the user doing things like examining verbose TODO output and writing lame tests. For every 1 edge case caught by making passing TODO tests fail you will cause 99 fully functional modules to fail. False negatives can be as bad as false positives. --->>>*** The primary users of CPAN are not developers ***<<<--- Most people type in "install Foo::Bar" into the CPAN shell and expect things to whiz by and at the end their module is magically installed. Most people who install CPAN modules are not CPAN testers. They're not developers. They're not savvy. They might not even know Perl, for instance a sysadmin fulfilling a user request to install a CPAN module. They're not interested in fixing the code and often wouldn't know how. They don't want to grovel through test failure output to figure out what went wrong (or what went right). They just want it to work. They don't want to be used as a giant beta testing system. By hanging out with developers all day you tend to forget this. If you're the 1 in a 100 of CPAN downloaders who want to examine test output to find passing TODO tests and report them back... great! Love to hear from ya. Flip a flag so they're displayed prominently for you. Don't cause inconvenience to 99 people so 1 person can report a fixed bug back to the author. It has to work... because most folks want to use the code, not fix it. [1] He was talking about the ultra-portability and ubiquity of Perl. Perl doesn't work everywhere, but we consider that a problem and that's important.
