Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?

2007-12-04 Thread Michael G Schwern
I'm going to sum up this reply, because it got long but kept on the same themes. * TODO tests provide you with information about what tests the author decided to ignore. ** Commented out tests provide you with NO information. ** Most TODO tests would have otherwise been commented out. * How y

Re: Why not run a test without a plan?

2007-12-04 Thread Michael G Schwern
Eric Wilhelm wrote: >> A. Pagaltzis wrote: >>> ... >>> which would still be an error. That way a mistake in a test >>> script won’t lead to Test::More silently converting an up-front >>> plan declarations into trailing ones. >> Which brings us back to the original question: why should that be an >

Re: Why not run a test without a plan?

2007-12-04 Thread Chris Dolan
On Dec 4, 2007, at 9:06 AM, David Golden wrote: On Dec 4, 2007 12:40 AM, Chris Dolan <[EMAIL PROTECTED]> wrote: Or maybe not reinvent the wheel and get tons of extra functionality for free! use base 'Test::Class'; sub block : Test(2) { ok(1, "wibble"); ok(1, "wobble"); } But if it goe

Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?

2007-12-04 Thread Fergal Daly
On 05/12/2007, Michael G Schwern <[EMAIL PROTECTED]> wrote: > This this whole discussion has unhinged a bit from reality, maybe you can give > some concrete examples of the problems you're talking about? You obviously > have some specific breakdowns in mind. I don't, I'm arguing against what has

Re: Why not run a test without a plan?

2007-12-04 Thread Michael G Schwern
Geoffrey Young wrote: > I guess what I thought you were getting at was a natural decoupling of > comparison functions with the planning without all the hackery involved > to get that sepraration working now. so I was suggesting that the > decoupling go further than just no_plan, and that yeah, roc

Re: Why not run a test without a plan?

2007-12-04 Thread Eric Wilhelm
# from Michael G Schwern # on Tuesday 04 December 2007 15:24: >A. Pagaltzis wrote: >>... >> which would still be an error. That way a mistake in a test >> script won’t lead to Test::More silently converting an up-front >> plan declarations into trailing ones. > >Which brings us back to the origina

Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?

2007-12-04 Thread Michael G Schwern
This this whole discussion has unhinged a bit from reality, maybe you can give some concrete examples of the problems you're talking about? You obviously have some specific breakdowns in mind. Fergal Daly wrote: >> Modules do not have a binary state of working or not working. They're >> compose

Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?

2007-12-04 Thread Eric Wilhelm
# from Fergal Daly # on Tuesday 04 December 2007 15:12: >I'm talking about people converting tests that were working just fine >to be TODO tests because the latest version of Foo (an external >module) has a new bug. While Foo is broken, they don't want lots of >bug reports from CPAN testers that t

Re: Why not run a test without a plan?

2007-12-04 Thread Geoffrey Young
Michael G Schwern wrote: > Geoffrey Young wrote: >> Andy Armstrong wrote: >>> On 4 Dec 2007, at 15:22, Geoffrey Young wrote: it would be nice if this were enforced on the TAP-digestion side and not from the TAP-emitter side - the coupling of TAP rules within the TAP-emitter is what

Re: UNKNOWN despite only failing tests -- how come?

2007-12-04 Thread Michael G Schwern
Andreas J. Koenig wrote: > Bug in CPAN::Reporter and/or Test::Harness and/or CPAN.pm? > > http://www.nntp.perl.org/group/perl.cpan.testers/796974 > http://www.nntp.perl.org/group/perl.cpan.testers/825449 > > All tests fail but Test::Harness reports NOTESTS and CPAN::Reporter > concludes UNKNO

Re: Why not run a test without a plan?

2007-12-04 Thread Michael G Schwern
Geoffrey Young wrote: > > Andy Armstrong wrote: >> On 4 Dec 2007, at 15:22, Geoffrey Young wrote: >>> it would be nice if this were enforced on the TAP-digestion side and not >>> from the TAP-emitter side - the coupling of TAP rules within the >>> TAP-emitter is what lead to my trouble in the firs

Re: Why not run a test without a plan?

2007-12-04 Thread Michael G Schwern
A. Pagaltzis wrote: > That would work. Of course once you have that, you don’t need to > allow assertions to run without a plan, since one can always say > > use Test::More tests => variable => 0; > pass(); > plan add_tests => 2; > pass(); > > instead of > > use Test::More; >

Re: shouldn't "UNEXPECTEDLY SUCCEEDED" mean failure?

2007-12-04 Thread Fergal Daly
On 02/12/2007, Michael G Schwern <[EMAIL PROTECTED]> wrote: > Fergal Daly wrote: > >> As long as you're releasing a new version, why would you not upgrade your > >> module's dependency to use the version that works? > > > > Your module either is or isn't usable with version X of Foo. > > > > If it

UNKNOWN despite only failing tests -- how come?

2007-12-04 Thread Andreas J. Koenig
Bug in CPAN::Reporter and/or Test::Harness and/or CPAN.pm? http://www.nntp.perl.org/group/perl.cpan.testers/796974 http://www.nntp.perl.org/group/perl.cpan.testers/825449 All tests fail but Test::Harness reports NOTESTS and CPAN::Reporter concludes UNKNOWN and CPAN.pm then installs it. Your

Re: UNKNOWN despite only failing tests -- how come?

2007-12-04 Thread David Golden
CPAN::Reporter says explicitly that UNKNOWN will return success to CPAN.pmand not prevent installation. I think this is a Test::Harness bug. David On Dec 4, 2007 3:24 PM, Andreas J. Koenig < [EMAIL PROTECTED]> wrote: > Bug in CPAN::Reporter and/or Test::Harness and/or CPAN.pm? > > http://www.n

Re: Why not run a test without a plan?

2007-12-04 Thread Geoffrey Young
Andy Armstrong wrote: > On 4 Dec 2007, at 15:22, Geoffrey Young wrote: >> it would be nice if this were enforced on the TAP-digestion side and not >> from the TAP-emitter side - the coupling of TAP rules within the >> TAP-emitter is what lead to my trouble in the first place. > > > A valid plan

Re: Why not run a test without a plan?

2007-12-04 Thread Andy Armstrong
On 4 Dec 2007, at 15:22, Geoffrey Young wrote: it would be nice if this were enforced on the TAP-digestion side and not from the TAP-emitter side - the coupling of TAP rules within the TAP-emitter is what lead to my trouble in the first place. A valid plan - at the beginning or the end - is

Re: Why not run a test without a plan?

2007-12-04 Thread Geoffrey Young
> Since the technical restriction is gone, and I see no particular benefit to it > being there, and it eliminates some tricky plan counting situations, I don't > see why it shouldn't be removed. this would be great, and help with situations like this: http://www.nntp.perl.org/group/perl.qa/200

Re: Why not run a test without a plan?

2007-12-04 Thread David Golden
On Dec 4, 2007 12:40 AM, Chris Dolan <[EMAIL PROTECTED]> wrote: > Or maybe not reinvent the wheel and get tons of extra functionality > for free! > > use base 'Test::Class'; > sub block : Test(2) { > >ok(1, "wibble"); >ok(1, "wobble"); > } But if it goes into Test::More, it eventually goes

Re: Why not run a test without a plan?

2007-12-04 Thread A. Pagaltzis
* Michael G Schwern <[EMAIL PROTECTED]> [2007-12-04 11:05]: > A. Pagaltzis wrote: > >* Michael G Schwern <[EMAIL PROTECTED]> [2007-12-04 03:35]: > >> It also makes it technically possible to allow the test to > >> change it's plan mid-stream > > > > Without some hypothetical future version of TAP

Re: Why not run a test without a plan?

2007-12-04 Thread Chris Dolan
On Dec 3, 2007, at 10:37 PM, Eric Wilhelm wrote: # from David Golden # on Monday 03 December 2007 19:55: With some sugar, that could actually be quite handy for something like test blocks. E.g.: { plan add => 2; ok( 1, "wibble" ); ok(1, "wobble" ); } or maybe make the block a sub

RE: Why not run a test without a plan?

2007-12-04 Thread Eric Harrison
I would find something along those lines incredibly helpful. Most of our tests have a variable number of tests based on a number of variables and so I use no_plan and every single test. Being able to add to the plan within a block of code would be massivley helpful and would allow us to be a

Re: Why not run a test without a plan?

2007-12-04 Thread Michael G Schwern
Smylers wrote: >>> It also makes it technically possible to allow the test to change >>> it's plan mid-stream >> Without some hypothetical future version of TAP this is only possible >> if you have run tests before declaring a plan at all, because >> otherwise the plan will already have been output

Re: Why not run a test without a plan?

2007-12-04 Thread Michael G Schwern
A. Pagaltzis wrote: > Yes, so this should be allowed: > > pass(); > plan 'no_plan'; > pass(); > > Whereas this should not: > > pass(); > plan tests => 2; > pass(); Umm, why not? That's exactly what I was proposing and it would result in... ok 1 ok 2

Re: Why not run a test without a plan?

2007-12-04 Thread Smylers
A. Pagaltzis writes: > Hi Michael, > > * Michael G Schwern <[EMAIL PROTECTED]> [2007-12-04 03:35]: > > > use Test::More; > > pass(); > > plan tests => 2; > > pass(); > > > > Why shouldn't this work? > > ... this should be allowed: > > pass(); > plan 'no_plan'; > pass(); >

Re: Why not run a test without a plan?

2007-12-04 Thread A. Pagaltzis
Hi Michael, * Michael G Schwern <[EMAIL PROTECTED]> [2007-12-04 03:35]: > use Test::More; > pass(); > plan tests => 2; > pass(); > > Why shouldn't this work? Currently you get a "You tried to run > a test without a plan" error, but what is it really protecting > the test author from? > >