Re: running tests

2004-04-08 Thread Ken Williams
On Apr 2, 2004, at 4:59 PM, Andy Lester wrote: Sure, but even better is to run only the tests that need to be run, which is a key part of prove. You can run prove -Mblib t/mytest.t instead of the entire make test suite. When I'm using Module::Build, I do this: Build test --test_files

Re: running tests

2004-04-05 Thread darren chamberlain
* Andy Lester andy at petdance.com [2004/04/02 16:59]: Sure, but even better is to run only the tests that need to be run, which is a key part of prove. You can run prove -Mblib t/mytest.t instead of the entire make test suite. $ make test TEST_FILES=t/mytest.t (darren) -- An idea is not

Re: running tests

2004-04-05 Thread Andy Lester
$ make test TEST_FILES=t/mytest.t Sure, and you can turn on HARNESS_VERBOSE to get the raw output of the .t file. prove puts all that stuff behind easy command-line switches, and lets you specify wildcards, and lets you specify a directory that implicitly does all the *.t within the

Re: running tests

2004-04-05 Thread Simon Cozens
[EMAIL PROTECTED] (Andy Lester) writes: Sure, and you can turn on HARNESS_VERBOSE to get the raw output of the .t file. prove puts all that stuff behind easy command-line switches, and lets you specify wildcards, and lets you specify a directory that implicitly does all the *.t within the

Re: running tests

2004-04-05 Thread Mark Stosberg
On Mon, Apr 05, 2004 at 05:05:34PM +0100, Simon Cozens wrote: [EMAIL PROTECTED] (Andy Lester) writes: Sure, and you can turn on HARNESS_VERBOSE to get the raw output of the .t file. prove puts all that stuff behind easy command-line switches, and lets you specify wildcards, and lets you

Re: running tests

2004-04-05 Thread Andy Lester
there's 'prove -h' and 'perldoc prove', making things much easier to learn and lookup. And, for that matter, prove --man == perldoc prove xoa -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance

Re: running tests

2004-04-03 Thread Fergal Daly
level). Also when running tests interactively it's nice to be able to save even 30 seconds by killing the suite if a low level test fails, Sure, but even better is to run only the tests that need to be run, which is a key part of prove. You can run prove -Mblib t/mytest.t instead of the entire

Re: running tests

2004-04-03 Thread Fergal Daly
On Sat, Apr 03, 2004 at 01:37:03AM +0200, Paul Johnson wrote: Coming soon to Devel::Cover (well, on my TODO list anyway): - Provide an optimal test ordering as far as coverage is concerned - ie tests which provide a large increase in coverage in a short time are preferred. There

running tests

2004-04-02 Thread Tim Harsch
Hi all, If I have several test files in my test suite, is there a way to get them to run in a predefined order when the user runs make test? I realize I could name them alphabetically like Atest1.t, Bsometest.t, but it seems hokey and I'm not sure it would work on all systems.

Re: running tests

2004-04-02 Thread Andy Lester
If I have several test files in my test suite, is there a way to get them to run in a predefined order when the user runs make test? I realize I could name them alphabetically like Atest1.t, Bsometest.t, but it seems hokey and I'm not sure it would work on all systems. Look at Test::Manifest

Re: running tests

2004-04-02 Thread Tim Harsch
PROTECTED] Cc: Perl Mod Authors [EMAIL PROTECTED] Sent: Friday, April 02, 2004 9:59 AM Subject: Re: running tests If I have several test files in my test suite, is there a way to get them to run in a predefined order when the user runs make test? I realize I could name them alphabetically like

Re: running tests

2004-04-02 Thread Arthur Corliss
On Fri, 2 Apr 2004, Tim Harsch wrote: Hi all, If I have several test files in my test suite, is there a way to get them to run in a predefined order when the user runs make test? I realize I could name them alphabetically like Atest1.t, Bsometest.t, but it seems hokey and I'm not sure it

Re: running tests

2004-04-02 Thread Tim Harsch
That seems a better idea than A, B etc. I'll just use that. Thanks! - Original Message - From: Arthur Corliss [EMAIL PROTECTED] To: Tim Harsch [EMAIL PROTECTED] Cc: Perl Mod Authors [EMAIL PROTECTED] Sent: Friday, April 02, 2004 10:32 AM Subject: Re: running tests On Fri, 2 Apr 2004

Re: running tests

2004-04-02 Thread Andy Lester
No. But there are certain classes of functions of the module that don't work until others have been run. So others should have been tested So some tests are setting up other ones, then? One of my goals when writing tests is to make everything as independent as possible, so that I can run a

Re: running tests

2004-04-02 Thread Mark Stosberg
On Fri, Apr 02, 2004 at 01:52:12PM -0600, Andy Lester wrote: No. But there are certain classes of functions of the module that don't work until others have been run. So others should have been tested So some tests are setting up other ones, then? One of my goals when writing tests is

Re: running tests

2004-04-02 Thread Andy Lester
For example, with DBD::Pg, a lot of tests depend on having test data in the database, and having the database connection working and open. Every one of our *.t and *.phpt files is self-contained. If it needs a connection to the database, it opens one. If it needs test data in the database,

Re: running tests

2004-04-02 Thread Tim Harsch
, April 02, 2004 11:52 AM Subject: Re: running tests No. But there are certain classes of functions of the module that don't work until others have been run. So others should have been tested So some tests are setting up other ones, then? One of my goals when writing tests is to make

Re: running tests

2004-04-02 Thread Mark Stosberg
On Fri, Apr 02, 2004 at 02:02:24PM -0600, Andy Lester wrote: For example, with DBD::Pg, a lot of tests depend on having test data in the database, and having the database connection working and open. Every one of our *.t and *.phpt files is self-contained. If it needs a connection to

Re: running tests

2004-04-02 Thread Andy Lester
Does that mean the test scripts are full of copy/paste coding? So if there is a bug in the test up routine, it would be propagated everywhere. That is indeed potentially the case. OTOH, once the code works, then changes to it are intentionally painful. It seems reasonable to break with the

Re: running tests

2004-04-02 Thread Mark Stosberg
On Fri, Apr 02, 2004 at 02:12:46PM -0600, Andy Lester wrote: I don't know what you mean by using prove? prove is a command-line utility that ships with Test::Harness. It allows you to run a specific test or tests, as specified on the command line, without having to go through the make test

Re: running tests

2004-04-02 Thread Paul Johnson
On Fri, Apr 02, 2004 at 02:19:03PM -0600, Andy Lester wrote: Does that mean the test scripts are full of copy/paste coding? So if there is a bug in the test up routine, it would be propagated everywhere. That is indeed potentially the case. OTOH, once the code works, then changes to it

Re: running tests

2004-04-02 Thread Adrian Howard
On 2 Apr 2004, at 20:59, Mark Stosberg wrote: [snip] One idea would seem to be have a testing module that provides the setup and tear-down functionality. Then each individual test could load the testing module, and setup and teardown for itself. [snip] That would be my approach. If you want some

Re: running tests

2004-04-02 Thread Andy Lester
coded correctly. So it's desirable to see the results of the lower level tests first because running the higer level tests could be a waste of time. But how often does that happen? Why bother coding to optimize the failures? Besides, if you have a smokebot to run the tests for you, then you

Re: running tests

2004-04-02 Thread Tim Harsch
PROTECTED] To: Fergal Daly [EMAIL PROTECTED] Cc: Tim Harsch [EMAIL PROTECTED]; Perl Mod Authors [EMAIL PROTECTED] Sent: Friday, April 02, 2004 12:51 PM Subject: Re: running tests coded correctly. So it's desirable to see the results of the lower level tests first because running the higer

Re: running tests

2004-04-02 Thread Andy Lester
Beats me what a smokebot is. Presumably it's something I should know about. But I only have so many hours in the day to know about everything, so simple and not requiring effort is better, for me anyway. I think you'll find that having a smokebot adds to the simple and not requiring

Re: running tests

2004-04-02 Thread Mark Stosberg
On Fri, Apr 02, 2004 at 03:48:12PM -0600, Andy Lester wrote: A smokebot is a script that runs your test suite at regular intervals, kicked off by cron. It checks out a given CVS branch, and then runs the entire test suite. For us, it runs once an hour, and if any tests fail, the entire

Re: running tests

2004-04-02 Thread Andy Lester
smoke $@ /home/smoke/smoke.out 21 And what does the inside of this 'smoke' script look like? It's just prove. An FLR-specific version of prove, but it's just prove. xoa -- Andy Lester = [EMAIL PROTECTED] = www.petdance.com = AIM:petdance

Re: running tests

2004-04-02 Thread Fergal Daly
presumably run the tests (depends on the size of the suite I suppose) before a checkin and it's convenient to know that the first failure message you see if the most relevant (ie at the lowest level). Also when running tests interactively it's nice to be able to save even 30 seconds by killing

Re: running tests

2004-04-02 Thread Andy Lester
Even if you have a smoke bot, you presumably run the tests (depends on the size of the suite I suppose) before a checkin and it's convenient to know that the first failure message you see if the most relevant (ie at the lowest level). Also when running tests interactively it's nice to be able

Re: running tests

2004-04-02 Thread Andy Lester
On Sat, Apr 03, 2004 at 01:37:03AM +0200, Paul Johnson ([EMAIL PROTECTED]) wrote: Coming soon to Devel::Cover (well, on my TODO list anyway): Could we pleease get it to run under -T first, though? Then I could do coverage testing on Test::Harness! xoa -- Andy Lester = [EMAIL PROTECTED] =