Thanks very much for the explanation,
Tried the following to get the Test/TAP crying...
use Test;
is 1, 1, 'test 1';
say "ok 1 - pestering 1";
say "nok 5 - pestering 2";
done-testing;
It then generates
ok 1 - test 1
ok 1 - pestering 1
nok 5 - pestering 2
1..1
Failed -1/1 subtests
Test Summary Report
---
confuse-TAP.t (Wstat: 0 Tests: 2 Failed: 0)
Parse errors: Tests out of sequence. Found (1) but expected (2)
Bad plan. You planned 1 tests but ran 2.
Files=1, Tests=2, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.15 cusr
0.01 csys = 0.19 CPU)
Result: FAIL
Looking back in the log I found a line with a test result at the end of
the line. This might have been missed by TAP.
2017-01-20 13:04:40.446841 [T] 1: socket found ok 2 - Scanned 200 docs,
bad searching
Printing to stderr does not help and I get the same problems with the
test above using 'note' instead of 'say'. It then comes down to
refraining from printing anything to prevent any mishap. Of course, it
is easy to prevent the 'ok ## - something' output but not when some real
ok output ends up at the end of some other line.
Btw. the perl version was retrieved from an installation on TRAVIS where
rakudobrew is used.
Regards,
Marcel
On 01/20/2017 10:04 PM, Zoffix Znet via RT wrote:
On Fri, 20 Jan 2017 09:16:18 -0800, mt1...@gmail.com wrote:
Hi Will,
How can it happen that a test gets hurt in this way? If I know this I
could better search for the problem.
Regards,
Marcel
Hi,
Here's how that error happens:
The TAP protocol[^1] expects tests to be numbered and (as seen in your output)
the test harness will complain if it spots tests missing or not in order.
So if your tests produce junk output (that is something other than stuff
produced by Test.pm6's routines), it could confuse the harness by making it
interpret junk output as a test.
Another harder way to accomplishing that is to run the tests in multiple
threads, to trigger a data race in the Test.pm6's internal test counter and
make it output wrong stuff.
Yet another way is to use a buggy test harness that fails to recognize a test
and so thinks something's missing. We had such a bug recently, though it got
introduced on Jan 2nd and got fixed[^2] on Jan 5th. From your version, I see
you're running Rakudo from some time in October, so this probably isn't it...
I'd suggest you try using a release version of Rakudo or HEAD. I tried running
that test on my box, but instead of TAP output it seems to be just producing
time-stamped connection failures.
[1] https://testanything.org/
[2]
https://github.com/rakudo/rakudo/commit/b120ac401a0795f5f4c5fcd6e775848b4b755508
