TODO Tests

2008-05-12 Thread Bram
[This idea was first submitted to p5p... See: http://www.nntp.perl.org/group/perl.perl5.porters/2008/05/msg136540.html ] While adding some todo tests (for t/op/range.t) I ran into some limitations (IMHO). Assume the following TODO test: { local $TODO = test that foo() returns 1;

Re: TODO Tests

2008-05-12 Thread Ovid
--- Bram [EMAIL PROTECTED] wrote: { local $TODO = test that foo() returns 1; local $WAS = 3; my $s1 = foo(); is($s1, 1); } I really haven't thought through all of the ramifications of this, but I like it. I was recently complaining to Schwern about the issue with TODO tests

Re: TODO Tests

2008-05-12 Thread Aristotle Pagaltzis
* Ovid [EMAIL PROTECTED] [2008-05-12 11:35]: Alternatively, persistent TAP could potentially track TODO results and handle the $WAS for you, but this is quite a ways off and has the problem that we cannot always identify which tests are which. Plus, you still need a way to specify which

Re: TODO Tests

2008-05-12 Thread Smylers
Bram writes: At the moment foo() returns 3. Time passes and code changes. Now there are 3 options: foo() returns 1, this will result in 'unexpected todo test passed' being outputted; foo() returns 3, no special output is produced; foo() returns 4, no special output is produced;

Re: TODO Tests

2008-05-12 Thread Ovid
--- Smylers [EMAIL PROTECTED] wrote: If you believe that (until the TODO is done) foo will consistently return 3, and you wish to be alerted if it suddenly starts returning 4, then surely you can do that with a non-TODO test which checks for its being 3? Sure you can do that: my $result;

Re: TODO Tests

2008-05-12 Thread Bram
Quoting Smylers [EMAIL PROTECTED]: Bram writes: At the moment foo() returns 3. Time passes and code changes. Now there are 3 options: foo() returns 1, this will result in 'unexpected todo test passed' being outputted; foo() returns 3, no special output is produced; foo() returns 4, no

Re: TODO Tests

2008-05-12 Thread Fergal Daly
2008/5/12 Ovid [EMAIL PROTECTED]: --- Smylers [EMAIL PROTECTED] wrote: If you believe that (until the TODO is done) foo will consistently return 3, and you wish to be alerted if it suddenly starts returning 4, then surely you can do that with a non-TODO test which checks for its being 3?

Re: TODO Tests

2008-05-12 Thread Ovid
--- Fergal Daly [EMAIL PROTECTED] wrote: I almost posted this a few hours ago but then decided not to since I'm not sure I like the thread at all. I'm posting it now because while I'm not a huge fan of the idea itself, the problems you list are due to a lazy interpretation of the idea (just

Re: TODO Tests

2008-05-12 Thread nadim khemir
On Monday 12 May 2008 16.23.46 Bram wrote: Then what happens if it starts returning 4? Then the test script will report a FAIL, and users will/might start ignoring failures. Which is a bad thing (IMHO). The todo test indicates that something doesn't behave as it should. If it suddenly

Re: TODO Tests

2008-05-12 Thread Bram
Quoting nadim khemir [EMAIL PROTECTED]: On Monday 12 May 2008 16.23.46 Bram wrote: Then what happens if it starts returning 4? Then the test script will report a FAIL, and users will/might start ignoring failures. Which is a bad thing (IMHO). The todo test indicates that something doesn't

Re: TODO Tests

2008-05-12 Thread chromatic
On Monday 12 May 2008 11:41:58 Bram wrote: What I'm suggesting is that it outputs something similar for TODO test   that return an unexpected result. TODO means if it's anything but this, tell me. You're trying to extract two bits of information from one bit of data. Zombie Claude Shannon

Re: TODO Tests

2008-05-12 Thread Nicholas Clark
On Mon, May 12, 2008 at 08:41:58PM +0200, Bram wrote: I'm not saying that it shouldn't change. What I'm saying is that the value may change but that if it changes I would like to be informed about it. Leave your todo test as it was to start with. Create a new test file

Re: TODO Tests

2008-05-12 Thread nadim khemir
On Monday 12 May 2008 20.41.58 Bram wrote: Leave your todo test as it was to start with. Create a new test file development_values_that_shoulnot_change.t in your developer test directory (that's not just for pod right). Which is not practical at all. That means maintaining an extra test

Re: Interrest for a QA-Tool hackathon after YAPC::EU

2008-05-12 Thread Thomas Klausner
Hi! On Tue, May 06, 2008 at 12:49:05PM +0200, nadim khemir wrote: In Oslo I proposed to organize a hackathon the week after YAPC::EU. Maybe you should add the hackathon to the YAPC::Europe 2008 wiki: http://www.yapceurope2008.org/ye2008/wiki -- #!/usr/bin/perl

Test::Harness 3.10 lots of issues under Devel::Cover 0.64

2008-05-12 Thread Gabor Szabo
Hi Paul, I am not sure if this was discussed already but while Test::Harness 3.10 passes its tests on my Ubuntu 7.10 with perl 5.8.8 it fails when running with Devel::Cover: Here is the beginning of the 2684 lines of output: Deleting database

libwww-perl-5.812 and Devel::Cover 0:64

2008-05-12 Thread Gabor Szabo
After running the following as I have been doing with many other modules cover -delete export DEVEL_COVER_OPTIONS=-coverage,statement,branch,condition,path,subroutine,time HARNESS_PERL_SWITCHES=-MDevel::Cover make test cover -report html all tests PASS and then I get: Can't open database