Geoffrey Young wrote:
$ perl -MTest::Harness -wle 'runtests @ARGV' ~/tmp/stdout.t
/Users/schwern/tmp/stdout....dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 1
Failed 1/1 tests, 0.00% okay
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
/Users/schwern/tmp/stdout.t 1 256 1 1 100.00% 1
Failed 1/1 test scripts, 0.00% okay. 1/1 subtests failed, 0.00% okay.
Oh look, no diagnostics at all.
um, ok, so I haven't quite finished my coffee this morning, but isn't
that what we would expect without $Test::Harness::verbose being set?
Have more coffee. It has never worked that way.
$Test::Harness::verbose is about showing the raw output of the .t file. This means all the
"ok" and "not ok" lines which it normally obscures. It only effects stuff
going to STDOUT. Things going to STDERR have always been displayed.
Here's a simple example which you can try while brewing up a pot.
$ cat fail.t
#!/usr/bin/perl -w
use Test::More tests => 1;
is 23, 42;
$ perl -MTest::Harness -wle 'runtests @ARGV' fail.t
fail....
fail....NOK 1# Failed test in fail.t at line 5.
# got: '23'
# expected: '42'
# Looks like you failed 1 test of 1.
fail....dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 1
Failed 1/1 tests, 0.00% okay
Failed Test Stat Wstat Total Fail List of Failed
-------------------------------------------------------------------------------
fail.t 1 256 1 1 1
Failed 1/1 test scripts. 1/1 subtests failed.
Files=1, Tests=1, 0 wallclock secs ( 0.02 cusr + 0.01 csys = 0.03 CPU)
Failed 1/1 test programs. 1/1 subtests failed.
I
certainly don't want diagnostics clogging up my output unless I ask for
them.
Its been doing that for the last 10 years or so. Try an espresso.
Yes, you do want failure diagnostics. Otherwise you're left not knowing why things
failed and we might as well go back to ed-style error reporting (ie. just print a
"?").
You get into fun situations where users report test failures with no diagnostic info and
you have to go tell them to rerun the tests with the verbose flag on. Or you run the
tests and get a failure and then you have to rerun them again with "verbose" on
to find out why (and its a heisenbug so it doesn't appear that time).
$ perl -MTest::Harness -wle '$Test::Harness::verbose = 1; runtests
@ARGV' /tmp/stdout.t
Test::Harness throws out all non-TAP stuff going to STDOUT. This
includes comments. So if Test::Builder started sending its diagnostics
to STDOUT they'd disappear into the ether.
I haven't found that to be true I don't think - because STDERR is the
error_log file, Apache-Test sends everything to STDOUT and it works just
fine for us. or so it has seemed for a while now...
Apache::Test, by default, sends diagnostics to STDERR. This is because by
default it uses Test.pm which sends its errors to STDERR.
$ perl -Mblib -wle 'use Apache::Test; plan tests => 1; ok 1, 2' > /dev/null
# Test 1 got: "1" (-e at line 1)
# Expected: "2"
Only when you explicitly say -withtestmore does it go to STDOUT.
$ perl -Mblib -wle 'use Apache::Test qw(-withtestmore); plan tests => 1; is 1, 2'
> /dev/null
$
-withtestmore seems like it changes the behavior of Apache::Test and seems a
little broken so I doubt anyone's using it, the tests don't, which is why you
haven't noticed.
$ perl -Mblib -wle 'use Apache::Test qw(-withtestmore); plan tests => 1; ok 1,
2'
1..1
# Using Apache/Test.pm version 1.28
# You named your test '2'. You shouldn't use numbers for your test names.
# Very confusing.
ok 1 - 2
