Re: Readd use of TAP subtests

2022-04-19 Thread Daniel Gustafsson
> On 19 Apr 2022, at 21:07, Jacob Champion wrote: > > On Mon, 2022-02-28 at 17:02 +0100, Peter Eisentraut wrote: >> Perhaps in another 7 years or so this will be resolved and we can make >> another attempt at this. ;-) > > For what it's worth, the TAP 14 spec was officially released today:

Re: Readd use of TAP subtests

2022-04-19 Thread Jacob Champion
On Mon, 2022-02-28 at 17:02 +0100, Peter Eisentraut wrote: > Perhaps in another 7 years or so this will be resolved and we can make > another attempt at this. ;-) For what it's worth, the TAP 14 spec was officially released today: https://testanything.org/tap-version-14-specification.html

Re: Readd use of TAP subtests

2022-02-28 Thread Peter Eisentraut
On 25.02.22 17:26, Andres Freund wrote: Ok that's good to know. What exactly happens when it tries to parse them? Does it not count them or does it fail somehow? The way the output is structured Says that it can't pase a line of the tap output: Ok, then I suppose I'm withdrawing this.

Re: Readd use of TAP subtests

2022-02-25 Thread Andrew Dunstan
On 2/25/22 11:41, Andres Freund wrote: > Hi, > > On 2022-02-25 09:43:20 -0500, Andrew Dunstan wrote: >> AIUI TAP consumers are supposed to ignore lines they don't understand. > Are they? > > In http://testanything.org/tap-version-13-specification.html there's: > > "Lines written to standard

Re: Readd use of TAP subtests

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 09:43:20 -0500, Andrew Dunstan wrote: > AIUI TAP consumers are supposed to ignore lines they don't understand. Are they? In http://testanything.org/tap-version-13-specification.html there's: "Lines written to standard output matching /^(not )?ok\b/ must be interpreted as test

Re: Readd use of TAP subtests

2022-02-25 Thread Jacob Champion
On Fri, 2022-02-25 at 16:35 +, Jacob Champion wrote: > On Fri, 2022-02-25 at 09:43 -0500, Andrew Dunstan wrote: > > AIUI TAP consumers are supposed to ignore lines they don't understand. > > It's undefined behavior [1]: And of course the minute I send this I notice that I've linked the v13

Re: Readd use of TAP subtests

2022-02-25 Thread Jacob Champion
On Fri, 2022-02-25 at 09:43 -0500, Andrew Dunstan wrote: > AIUI TAP consumers are supposed to ignore lines they don't understand. It's undefined behavior [1]: > Any output that is not a version, a plan, a test line, a YAML block, > a diagnostic or a bail out is incorrect. How a harness handles

Re: Readd use of TAP subtests

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 14:39:15 +0100, Peter Eisentraut wrote: > On 24.02.22 16:00, Andres Freund wrote: > > I've incidentally played with subtests yesterdays, when porting > > src/interfaces/libpq/test/regress.pl to a tap test. Unfortunately it seems > > that subtests aren't actually specified in the

Re: Readd use of TAP subtests

2022-02-25 Thread Andrew Dunstan
On 2/25/22 08:39, Peter Eisentraut wrote: > On 24.02.22 16:00, Andres Freund wrote: >> I've incidentally played with subtests yesterdays, when porting >> src/interfaces/libpq/test/regress.pl to a tap test. Unfortunately it >> seems >> that subtests aren't actually specified in the tap format,

Re: Readd use of TAP subtests

2022-02-25 Thread Peter Eisentraut
On 24.02.22 16:00, Andres Freund wrote: I've incidentally played with subtests yesterdays, when porting src/interfaces/libpq/test/regress.pl to a tap test. Unfortunately it seems that subtests aren't actually specified in the tap format, and that different libraries generate different output

Re: Readd use of TAP subtests

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 11:16:03 +0100, Peter Eisentraut wrote: > Now that we have switched everything to done_testing(), the subtests feature > isn't that relevant anymore, but it might still be useful to get better > output when running with PROVE_FLAGS=--verbose. I've incidentally played with

Re: Readd use of TAP subtests

2022-02-24 Thread Daniel Gustafsson
> On 24 Feb 2022, at 11:16, Peter Eisentraut > wrote: > I think for deeply nested tests and test routines defined in other modules, > this helps structure the test output more like the test source code is laid > out, so it makes following the tests and locating failing test code easier. I

Re: Readd use of TAP subtests

2022-02-24 Thread Peter Eisentraut
source code is laid out, so it makes following the tests and locating failing test code easier.From 3f50bc5236d7793939904222a38f7e13a2cda47c Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Thu, 24 Feb 2022 11:01:47 +0100 Subject: [PATCH v2] Readd use of TAP subtests Since

Re: Readd use of TAP subtests

2021-12-09 Thread Daniel Gustafsson
> On 8 Dec 2021, at 16:25, Tom Lane wrote: > I think the main point is to make sure that the test script reached an > intended exit point, rather than dying early someplace. It's not apparent > to me why reaching a done_testing() call is a less reliable indicator of > that than executing some

Re: Readd use of TAP subtests

2021-12-09 Thread Peter Eisentraut
On 08.12.21 18:31, Tom Lane wrote: A question that seems pretty relevant here is: what exactly is the point of using the subtest feature, if we aren't especially interested in its effect on the overall test count? I can see that it'd have value when you wanted to use skip_all to control a

Re: Readd use of TAP subtests

2021-12-08 Thread Tom Lane
=?utf-8?Q?Dagfinn_Ilmari_Manns=C3=A5ker?= writes: > The only cases where an explicit plan adds value is if you're running > tests in a loop and care about the number of iterations, or have a > callback with a test inside that you want to make sure gets called. For > these, it's better to

Re: Readd use of TAP subtests

2021-12-08 Thread Tom Lane
Andrew Dunstan writes: > On 12/8/21 09:08, Dagfinn Ilmari Mannsåker wrote: >> Either way, I think we should be switching tests to done_testing() >> whenever it would otherwise have to adjust the test count, to avoid >> having to do that again and again and again going forward. > I'm not so sure.

Re: Readd use of TAP subtests

2021-12-08 Thread Dagfinn Ilmari Mannsåker
Andrew Dunstan writes: > On 12/8/21 09:08, Dagfinn Ilmari Mannsåker wrote: >> >> Either way, I think we should be switching tests to done_testing() >> whenever it would otherwise have to adjust the test count, to avoid >> having to do that again and again and again going forward. >> > > I'm not

Re: Readd use of TAP subtests

2021-12-08 Thread Andrew Dunstan
On 12/8/21 09:08, Dagfinn Ilmari Mannsåker wrote: > > Either way, I think we should be switching tests to done_testing() > whenever it would otherwise have to adjust the test count, to avoid > having to do that again and again and again going forward. > I'm not so sure. I don't think its

Re: Readd use of TAP subtests

2021-12-08 Thread Dagfinn Ilmari Mannsåker
Daniel Gustafsson writes: >> On 8 Dec 2021, at 14:49, Dagfinn Ilmari Mannsåker wrote: >> >> Peter Eisentraut writes: >> >>> Now that subtests in TAP are supported again, I want to correct the >>> great historical injustice of 7912f9b7dc9e2d3f6cd81892ef6aa797578e9f06 >>> and put those

Re: Readd use of TAP subtests

2021-12-08 Thread Andrew Dunstan
On 12/8/21 08:26, Peter Eisentraut wrote: > > Now that subtests in TAP are supported again, I want to correct the > great historical injustice of 7912f9b7dc9e2d3f6cd81892ef6aa797578e9f06 > and put those subtests back. > > Much more work like this is possible, of course.  I just wanted to get >

Re: Readd use of TAP subtests

2021-12-08 Thread Daniel Gustafsson
> On 8 Dec 2021, at 14:49, Dagfinn Ilmari Mannsåker wrote: > > Peter Eisentraut writes: > >> Now that subtests in TAP are supported again, I want to correct the >> great historical injustice of 7912f9b7dc9e2d3f6cd81892ef6aa797578e9f06 >> and put those subtests back. > > The updated

Re: Readd use of TAP subtests

2021-12-08 Thread Dagfinn Ilmari Mannsåker
Peter Eisentraut writes: > Now that subtests in TAP are supported again, I want to correct the > great historical injustice of 7912f9b7dc9e2d3f6cd81892ef6aa797578e9f06 > and put those subtests back. The updated Test::More version requirement also gives us done_testing() (added in 0.88), which

Readd use of TAP subtests

2021-12-08 Thread Peter Eisentraut
written and I've had this on my list for, uh, 7 years.From d1e2df592b7eaa7952d621266d5ad5d6b80e503a Mon Sep 17 00:00:00 2001 From: Peter Eisentraut Date: Wed, 8 Dec 2021 14:20:57 +0100 Subject: [PATCH] Readd use of TAP subtests Since 405f32fc49609eb94fa39e7b5e7c1fe2bb2b73aa, Test::More must be new