Ricardo SIGNES wrote:
> * Michael G Schwern <[EMAIL PROTECTED]> [2007-03-09T18:17:57]
>> *) TAP diagnostic format
>> http://perl-qa.yi.org/index.php/TAP_diagnostic_syntax
>>
>> There is no way to output diagnostics in TAP. The stuff Test::More spits
>> out to STDERR are unparsable comments indented for humans. Its not TAP.
>> This is the single biggest TAP deficiency.
>
> "Diagnostics need not be only for failures. There's nothing which says you
> can't output them for passes."
>
> So, under what circumstances do we display the diagnostics data regarding test
> line/file? Something like either:
>
> * if other diagnostics were given, we add these and display it
> * if the test fails, we add these and display it
(Talking with rjbs in chat discovered that when he says "display" here he
means the TAP producer outputting diagnostics, not the harness displaying them)
The TAP producer should probably spit out anything that might be useful
always. This means the file, line and name on all tests.
The nice part is that decision is entirely up to the displayer and we give
them enough information to decide. A tty oriented display which has only one
output stream will probably want to go with "display only diagnostics for
failing tests" by default but provide switches to see them all. Something
webby like the HTML matrix used by Pugs can show all diagnostics as part of a
tooltip when the user hovers over a link.
One concern is that diagnostics for passing tests will clutter up the raw TAP
output for those who like to read the stream of "ok" and "not ok".
Fortunately, with TAP::Parser, you can put together a filtered TAP reader
which can read in the raw TAP, strip out whatever you don't want to see, and
spit out the filtered TAP.
> Assuming that diagnostics are passed as a hashref, and someone passes "line"
> or
> "file" entries (or whatever the standard name is), presumably we croak. Or,
> we
> make the standard name obnoxious enough that by using it you are clearly
> stating that you know you're about to shoot your foot. ('TAP/testline')
Conflicting keywords... I'm inclined to say that the default case should be
that the standard keys should be the shorter and most readable since that's
going to be the majority case.
Guarding against a user passing in a file and line diagnostic with a different
meaning (maybe they mean the file they're currently reading from and the line
they're on) can be done by the TAP producer. That is, if they say:
is $line, "foo", { name => "the foo line", file => $file, line => $. };
Test::Builder can define "file" and "line" here to go under "extensions" so
the output might look like:
not ok 23 #---
got: "bar"
expected: "foo"
file: test.t
line: 68
name: the foo line
extensions:
file: my_file
line: 23
