* Andy Armstrong <[EMAIL PROTECTED]> [2008-08-22 00:35]: > Indeed - but there's a sweet spot somewhere between specifying > too little and specifying too much. It's entirely proper that > there should be a debate about where that sweet spot is - and > parsimony should be a guiding principle - but the value of > having inline, per-test metadata has already been demonstrated.
Fully and strongly agreed, ++ on all counts. > IMO we don't need a complete serialisation protocol - just > somewhere we can make notes and have them pass unmolested > through the harness. If someone needs arbitrary serialisation > they can use a protocol that makes sense to them and either > inline their serialised data as a Base64 encoded string or have > the diagnostic refer to an external file. The important thing > is to open up a conduit through which user defined data can > flow from a test script. […] I favour JSON because it's the > simplest solution that fits those criteria. Exactly. We want to make it easy to annotate tests with any sort of testing-relevant metadata that people might have, but the purpose of the protocol is supposed to be conveying test results, not one-way messaging of arbitrary data. Eg. someone who uses TAP in testing a web app will probably have use for an HTTP response status key and also for a hash of response headers. The killer app example for diagnostics that I always come back to is statical analysis over an archive of your TAP streams over time. It makes sense to preserve as much test-relevant metadata as possible to be able to examine trends. For this purpose, the data does have to be machine readable, and it does have to be structured at least a little – *just* a flat key-value bag won’t cut it. However, no intricate structure is necessary. This is a point where the APIs can help. If the API encourages people to indiscriminately dump complex data structures into the TAP stream, that’s what people will do. If the API instead makes it very easy to use diagnostics in the way we intend for them to be used, and doesn’t go out of its way to accomodate the potato bag usage, fewer people will be tempted. Regards, -- Aristotle Pagaltzis // <http://plasmasturm.org/>
