I've just noticed that the MediaWiki team have a test harness that
tracks the life of individual assertions over time:
* HTML nested bullet list, open tags (bug 5497) [Has never passed]
* HTML nested ordered list, open tags (bug 5497) [Has never passed]
* Fuzz testing: image with bogus manual thumbnail [Introduced
between 08-Apr-2007 07:15:22, 1.10alpha (r21099) and 25-Apr-2007
07:15:46, 1.10alpha
Nice eh?
With TAP that'd require either adding a persistent identifier to tests,
requiring that the test numbering doesn't change over time (new tests
added at the end) or that there's some heuristic that attempts to track
assertions that have been renumbered.
http://testanything.org/wiki/index.php/Assertions_are_named_dynamically
A partial solution may lie in the direction of Test Blocks[1]. If it
became normal practice for small clusters of related assertions to be
grouped in a block then the TAP document would gain structure that would
make it easier to automagically work out which assertions had been added
or removed. Test Blocks would also localise the impact on test numbering
when a new assertion is added; outside of the affected block the
numbering wouldn't change.
And I still like the Test Blocks proposal for other reasons...
[1] http://testanything.org/wiki/index.php/Test_Blocks
--
Andy Armstrong, Hexten
