On Dec 2, 2007, at 4:11 PM, Michael G Schwern wrote:
Fergal Daly wrote:
Another downside of using TODO like this is that when the external
module is fixed, you have to release a new version of your module
with
the TODOs removed. These tests will start failing for anyone who
upgrades your module but not broken one but in reality nothing has
changed for that user,
As long as you're releasing a new version, why would you not
upgrade your
module's dependency to use the version that works?
Yep, that's it exactly. See, Perl::Critic's problem was that the PPI
SVN repository had important bug fixes for a almost a full year
before and actual PPI release was pushed to CPAN. Some of those bugs
caused significant false negatives in Perl::Critic's code checks.
The test was right, but the code was wrong which sure seems like a
perfect definition of a TODO test. I personally was developing
against the SVN PPI and adding TODOs and workarounds to support the
older version.
Since PPI 1.200 was released with all of the bug fixes and we
released a Perl::Critic that requires 1.200, I fully agree that we
*should* have removed the TODO flag on those tests. But there's
always other work to be done... Patches welcome. :-)
The problem with skipped tests is that they're easier for developers
to ignore than TODO tests. And you have to now worry about three
things instead of two: is the code right, is the test right, and is
the skip conditional right. I've gotten the latter wrong before and
lean toward TODO tests as a result.
Chris
