On Monday 07 April 2008 14:42:26 Ovid wrote:
> The main problem we're trying to balance is the needs of complex test
> suites (less common) and simple test suites (more common). Creating a
> protocol which tries to satisfy both of those is tough, so we're also
> working on tool support to manage it.
>
> It's a more difficult problem because as we expand to fit more complex
> needs, the average CPAN author might get mad that we're adding support
> for things they don't need, but organizations like Yahoo, the BBC and
> quite a few others find that Perl's testing tools can be quite limiting
> for their needs. This doesn't help.
Sure, and I'm sympathetic to that argument. A little more structure is a good
thing, if there are tools which can read it unambiguously.
However, trying to design in sufficient flexibility that the toolset never
needs to be upgraded despite everyone everywhere throwing in new key/value
pairs willy-nilly misses, I think, one very important point:
All* of the TAP I've ever seen has been transient data generated by one
tool,
processed by another tool, and almost immediately discarded.
Let's make that two points:
TAP and Test::Builder succeed by making tests unambiguous. They passed
or
they didn't. True or false; that's it.
Universal and pain-free extensibility is a nice goal, I suppose, but I've
never actually seen it in practice. It usually fails because of explosive
ambiguity.
YAML parsers right now *should* ignore extra fields. If people want to add
extra fields, that's fine. They can write their own parsers. If TAP-diag
needs to add fields in the future, the existing tools can do so as well.
People who might want to upgrade then have the same choices they always do:
1) don't upgrade
2) upgrade but roll back any changes they don't want
3) upgrade and change their existing modifications to avoid collisions
(If you really want to start a free-for-all, throw in an officially
recommended "metadata" hash and laissez votre mains under that.)
Again, we're talking about processing documents generated and almost always
immediately thrown away after processing. Any organization which can't fix
the handful of pieces which generate TAP-diag and the one or two pieces of
code which consume TAP-diag in an afternoon is doing something wrong with
regard to duplication and maintainability, and it'll take quite a few
libations to make me believe that such organizations are actually testing
their code in any rigorous sense.
-- c
* except for TAP written by hand for didactic purposes, which I've seen three
times, and the first two don't count because I wrote it
