On Wednesday 14 March 2007 15:05, Michael G Schwern wrote: > chromatic wrote:
> > I think my point eluded everyone. Let me be very clear. > > > > MAKE diag() PRINT TO STDOUT NOT STDERR. > > Deja vu all over again. We had this discussion months ago when TAP::Parser > tried EXACTLY this, at the time by tricking Test::Builder's failure_output > method, and I explained it was a bad idea. > > Here's one. > http://www.mail-archive.com/perl-qa@perl.org/msg06694.html Merely skimming that message reveals a falsehood: Test::Harness throws out all non-TAP stuff going to STDOUT. This includes comments. So if Test::Builder started sending its diagnostics to STDOUT they'd disappear into the ether. That hasn't been true for *nearly two years*. I seem to recall patching Test::Harness::Straps before the coffee stain book came out in July 2005. > Here's another, referencing that first one. > http://www.nntp.perl.org/group/perl.qa/2006/09/msg7152.html > And yet another one. > http://www.nntp.perl.org/group/perl.qa/2006/09/msg7153.html > And that's just in September. I feel like I have to refute this idea about > once a month. Here we go again. I remember that thread now, the one where people like Ovid, Aristotle, David, Adrian, and chromatic all said "Uh, making one small change to Test::Builder and fixing Test.pm too would improve 99% of the Perl world without making life worse than it already is for anyone else". Did I miss your response to everything there? > Piping all diagnostics to STDOUT solves nothing except maybe allowing > runtests to display warnings again. You still can't tell the difference > between a comment (what currently is "# foo" printed to STDOUT) and a > failure diagnostic (what currently is "# foo" printed to STDERR) and > diagnostics associated with a TODO test (which is "# foo" printed to > STDOUT). Test::Harness can identify TODO tests. Test::Harness can even tell if a diagnostic followed a TODO test. > There must be a way to differentiate between "this is information to be > displayed to the user" and "this is information not to be displayed to the > user". Currently that distinction is STDERR vs STDOUT. There is currently > no other way for a harness to distinguish. Yes, that's precisely why I'm suggesting that there could be another way for a harness to distinguish! Let it continue to ignore STDERR! > Consider the following. Again. > > $ cat ~/tmp/foo.t > #!/usr/bin/perl -w > > $| = 1; > > print "1..2\n"; > print "ok 1\n"; > print "# This is not displayed\n"; > print "not ok 2\n"; > print STDERR "# This displayed.\n"; ... > $ runtests ~/tmp/foo.t > /Users/schwern/tmp/foo......1/2 # This is not displayed > /Users/schwern/tmp/foo......2/2 # This displayed. > /Users/schwern/tmp/foo...... Failed 1/2 subtests You didn't have to convince me that interleaving STDOUT and STDERR is a mistake. I already believed that. > There are proposals on the table for machine parsable diagnostics on the > wiki but nothing to indicate free-form information to be displayed to the > user. Currently: - the test harness displays all free-form information to the user by virtue of not touching it at all In the future: - the test harness could display all free-form information to the user by virtue of it being identifiably free-form information Currently: - no harness can take advantage of information sent to STDERR - the existing Test::Harness::Straps already knows how to handle diagnostic information! In the future: - TAP producers that continue to send diagnostics to STDERR are no worse off! - TAP consumers that use Test::Harness::Straps can handle diagnostic information Displaying ALL diagnostics is not only not harmful but it is consistent with Test::Harness::TAP: Diagnostics Additional information may be put into the testing output on separate lines. Diagnostic lines should begin with a "#", which the harness must ignore, at least as far as analyzing the test results. The har‐ ness is free, however, to display the diagnostics. > > Problems solved: > > > > - synchronization no longer an issue > Synching is still an issue as warnings will be out of sync. C'est le > guerre. Perl warnings? The ones that go to STDERR by default? How in the world do they go through diag()? > I will not have Test::Builder in bed with the harness. Gonna be awful difficult for them to understand each other if they don't, erm, speak the same TAP. > I'm unconvinced this is something we can just make go away with a clever > trick. And so the right solution is to figure out a clever, cross-platform, cross-version way to both merge and umerge the interleaved output of two separate filehandles while rejecting all semantics that could possibly disambiguate whether the output came from anywhere within the tests or something completely unrelated elsewhere? That is indeed not clever. -- c
