chromatic wrote: > On Wednesday 14 March 2007 09:41, Geoffrey Young wrote: > >>> There ought to be a way to capture diagnostic information in the TAP >>> stream because it's useful for diagnosing problems. > >> so, to chromatic's point, I can't help but feel like solving the quoted >> issue would go a long way toward reducing the costs in this debate. > > I think my point eluded everyone. Let me be very clear. > > MAKE diag() PRINT TO STDOUT NOT STDERR.
Deja vu all over again. We had this discussion months ago when TAP::Parser tried EXACTLY this, at the time by tricking Test::Builder's failure_output method, and I explained it was a bad idea. Here's one. http://www.mail-archive.com/perl-qa@perl.org/msg06694.html Here's another, referencing that first one. http://www.nntp.perl.org/group/perl.qa/2006/09/msg7152.html And yet another one. http://www.nntp.perl.org/group/perl.qa/2006/09/msg7153.html And that's just in September. I feel like I have to refute this idea about once a month. Here we go again. Piping all diagnostics to STDOUT solves nothing except maybe allowing runtests to display warnings again. You still can't tell the difference between a comment (what currently is "# foo" printed to STDOUT) and a failure diagnostic (what currently is "# foo" printed to STDERR) and diagnostics associated with a TODO test (which is "# foo" printed to STDOUT). There must be a way to differentiate between "this is information to be displayed to the user" and "this is information not to be displayed to the user". Currently that distinction is STDERR vs STDOUT. There is currently no other way for a harness to distinguish. Consider the following. Again. $ cat ~/tmp/foo.t #!/usr/bin/perl -w $| = 1; print "1..2\n"; print "ok 1\n"; print "# This is not displayed\n"; print "not ok 2\n"; print STDERR "# This displayed.\n"; $ prove ~/tmp/foo.t /Users/schwern/tmp/foo....# This displayed. /Users/schwern/tmp/foo....FAILED test 2 Failed 1/2 tests, 50.00% okay Failed Test Stat Wstat Total Fail List of Failed ------------------------------------------------------------------------------- /Users/schwern/tmp/foo.t 2 1 2 Failed 1/1 test scripts. 1/2 subtests failed. Files=1, Tests=2, 0 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU) Failed 1/1 test programs. 1/2 subtests failed. $ runtests ~/tmp/foo.t /Users/schwern/tmp/foo......1/2 # This is not displayed /Users/schwern/tmp/foo......2/2 # This displayed. /Users/schwern/tmp/foo...... Failed 1/2 subtests Test Summary Report ------------------- /Users/schwern/tmp/foo.t (Wstat: 0 Tests: 2 Failed: 1) Failed tests: 2 Files=1, Tests=2, 0 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU) There are proposals on the table for machine parsable diagnostics on the wiki but nothing to indicate free-form information to be displayed to the user. > Problems solved: > > - synchronization no longer an issue Synching is still an issue as warnings will be out of sync. C'est le guerre. > - non-escaped output no longer interfering with the TAP stream > - unrelated output no longer interfering with the TAP stream Well... from STDERR you mean. > - most test modules are *already* using diag() for diagnostics And a lot aren't. Some use Test.pm. Some print directly to STDERR. Some use TAP producers we've never heard of. I will not have Test::Builder in bed with the harness. > Test::Harness::Straps can already identify diagnostic output; it's a simple > matter of programming to print it along with consuming it. > > Why is this all so complicated? Because it is complicated. Because the currently TAP in practice is not designed to be a single stream. Because if we screw this up it effects thousands of people. I'm unconvinced this is something we can just make go away with a clever trick.
