--- chromatic <[EMAIL PROTECTED]> wrote: > (Given that even unstructured diagnostics have never actually > appeared in TAP > documents before, my guess is "No, everyone's arguing out of > ignorance",
This is not true at all. If you rephrase this slightly: Given that even unstructured diagnostics have never actually appeared in test documents before, my guess is "No, everyone's arguing out of ignorance", There's a lot of prior art in this area, just not in TAP. Perl's needs have been relatively primitive because Perl's tools have been relatively primitive. We've become comfortable eating gruel for breakfast because that's all we have. Take a look at TestNG, the ant XML test output or any other well-used non-TAP test information format and you can see that people can and do use this diagnostic information all the time. We really have no ability to offer a robust solution if we're offering Notepad and someone needs to output RTF. Further, we can learn from their mistakes. Consider this snippet of TestNG output (from http://testng.org/doc/documentation-main.html): <test-method status="FAIL" signature="test1()" name="test1" duration-ms="0" started-at="2007-05-28T12:14:37Z" description="someDescription2" finished-at="2007-05-28T12:14:37Z"> <exception class="java.lang.AssertionError"> <short-stacktrace>java.lang.AssertionError ... Removed 22 stack frames </short-stacktrace> </exception> </test-method> By reading through TestNG, we can see that it has a heavy bias towards OO programming, something that doesn't necessarily fit what TAP is trying to accomplish. Further, look at their started-at, finished-at and duration-ms attributes. Change that duration-ms attribute to have the value 4483. Clearly this conflicts with the start and end times. What the hell does that mean? This duplication of information is something we can learn from and avoid. That being said, there's a lot of stuff in there that someone who uses TestNG would want, but won't necessarily be for all types of testing. User supplied diagnostics gives people the ability to have the information they need (and heck, even translate TAP into TestNG XML format). Lots of folks at the 2006 Google Test Automation Conference in London kept talking about the need for a way of exchanging test information between different systems, but current implementations were inadequate in large part because of the assumptions they make. TAP makes too few assumptions and TestNG appears to make too many. We can provide a common ground of making common assumptions and allowing users to provide the bits they need. Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Perl and CGI - http://users.easystreet.com/ovid/cgi_course/ Personal blog - http://publius-ovidius.livejournal.com/ Tech blog - http://use.perl.org/~Ovid/journal/
