On 30/11/18 16:50, Daniel P. Berrangé wrote:
> IIUC, this output is only seen when you with 'make check' passing V=1,
> so most people won't see it.
V=1 should be enabled by default on Patchew, so it will be visible in
the CI logs. But anyway that's minor, there are more reasons why a
custom beautifier is better than prove.
First, at the very least, failures should be included in the output even
if V=1 is not provided. Therefore, for failing tests we'd have:
$ scripts/tap-driver.pl < f.tap
FAIL 2 bbb
XPASS 3 ccc # TODO
FAIL 6 fff # SKIP cannot find frobnicator
And the output of prove in that case would be inferior for developers,
unless you use "V=1":
$ prove -e cat f.tap
f.tap .. Failed 2/6 subtests
(less 2 skipped subtests: 2 okay)
(1 TODO test unexpectedly succeeded)
Test Summary Report
-------------------
f.tap (Wstat: 0 Tests: 6 Failed: 2)
Failed tests: 2, 6
TODO passed: 3
Files=1, Tests=6, 0 wallclock secs ( 0.02 usr + 0.00 sys = 0.02 CPU)
Result: FAIL
where you don't really know what tests 2 and 6 are without looking at
the TAP output. The part before the summary provides a nice progress
report, as you point out, but it adds a bunch of noise with respect to
skipped subtests (skipped tests will be more common once patch 1 is
merged; qgraph also adds some) that I personally dislike.
You can hide it with -Q, but then you cannot invoke anymore more than
one test with a single invocation of prove, and the Makefile output
becomes much bigger:
TEST check-foo
vs.
TEST check-foo
All tests successful.
Files=1, Tests=3, 0 wallclock secs ( 0.01 usr + 0.00 sys = 0.01 CPU)
Result: PASS
There are two other smaller "philosophical" differences. First, prove
treats XPASS as passes, while tap-driver.pl treats them as failures.
Second, skipped tests really stand out in the "prove -v" output, with
white-on-blue text.
I'd rather avoid getting into the business of beautifying the output of
prove.
Paolo
