Re: Comment about BAIL_OUT

2007-01-05 Thread David Landgren
Greg Sabino Mullane wrote: [...] [1] I've never had a need for random tests myself. The only reason I break mine apart is to isolate testing various sub-systems, but I almost always end up having some dependencies put into an early 00 file. I also tend to a have a final 99 cleanup file. While

Re: Comment about BAIL_OUT

2007-01-05 Thread Michael G Schwern
Ovid wrote: However, if you use the '-s' switch to shuffle your tests and bailout is not first, then some tests will run until the BAIL_OUT is hit. This seems to violate the principle that tests should be able to run in any order without dependencies. It doesn't violate the principle since

Desired test output?

2007-01-05 Thread Ovid
From http://www.perlmonks.org/?node_id=593087: I'm working on TAPx::Parser and trying very hard to make my TAPx::Harness output as similar to Test::Harness output as is feasible. I am doing this primarily because if the basic output is too different from what folks are used to, the strangeness

Re: Desired test output?

2007-01-05 Thread Eric Wilhelm
# from Ovid # on Friday 05 January 2007 01:50 am: TAPx::Parser collects far more information than Test::Harness, so if there's more stuff you'd like to see, that's fine, too. You could dump it all into some kind of data (yaml?) file, then execute $ENV{TAP_RESULTS_VIEWER} or something?

Re: Comment about BAIL_OUT

2007-01-05 Thread Greg Sabino Mullane
Michael G Shwern wrote: Such a bother. ... You can even get clever and pack the setup/teardown calls into loading the module so you have even less code per script. Now each test runs independently and cleans itself up. True, but at the expense of having to run the startup and cleanup code

Re: Comment about BAIL_OUT

2007-01-05 Thread Michael Peters
Greg Sabino Mullane wrote: Michael G Shwern wrote: Such a bother. ... You can even get clever and pack the setup/teardown calls into loading the module so you have even less code per script. Now each test runs independently and cleans itself up. True, but at the expense of having to

Thoughts about test harness summary

2007-01-05 Thread Ovid
Pursuant to some discussion with BrowserUK at http://perlmonks.org/?node_id=593087, I'm looking at this and seeing some problems. -- Failed Test Stat Wstat Total Fail List of Failed - t/bar.t4 1024134 2 6-8

Re: Thoughts about test harness summary

2007-01-05 Thread Nicholas Clark
On Fri, Jan 05, 2007 at 11:11:25AM -0800, Ovid wrote: Pursuant to some discussion with BrowserUK at http://perlmonks.org/?node_id=593087, I'm looking at this and seeing some problems. -- Failed Test Stat Wstat Total Fail List of Failed

Re: Thoughts about test harness summary

2007-01-05 Thread Andy Lester
On Jan 5, 2007, at 1:28 PM, Nicholas Clark wrote: Failed Test | Total | Fail | List of Failed | TODO Passed +---+--++ t/bar.t | 13| 4|2, 6-8 |3-4

Re: Thoughts about test harness summary

2007-01-05 Thread David Golden
On 1/5/07, Andy Lester [EMAIL PROTECTED] wrote: On Jan 5, 2007, at 1:28 PM, Nicholas Clark wrote: Failed Test | Total | Fail | List of Failed | TODO Passed +---+--++ t/bar.t | 13| 4|2, 6-8 |3-4

Re: Thoughts about test harness summary

2007-01-05 Thread Ovid
--- Nicholas Clark [EMAIL PROTECTED] wrote: I'd like Wstat, even if I don't have Stat. I like to know if tests coredumped. I may be in a minority here, but being able to optionally switch to that output is useful. Fair enough. I like the prominence of TODO passed Cool. I hated wedging it

Re: Thoughts about test harness summary

2007-01-05 Thread Ovid
--- David Golden [EMAIL PROTECTED] wrote: On 1/5/07, Andy Lester [EMAIL PROTECTED] wrote: On Jan 5, 2007, at 1:28 PM, Nicholas Clark wrote: Failed Test | Total | Fail | List of Failed | TODO Passed +---+--++ t/bar.t

Re: Thoughts about test harness summary

2007-01-05 Thread Andy Lester
On Jan 5, 2007, at 1:46 PM, Ovid wrote: Failed Test Total Fail List of FailedTODO Passed -- t/bar.t 13 9 2, 6-8, 13, 17, 33-35 3-4 t/foo.t 10 10 5, 19, 27, 37-38, 117 9-11 Did you mean

TAP::Tests

2007-01-05 Thread Ovid
Some of the limitations of TAPx::Parser are due to how Test::Builder works. One thing which isn't making it into 'runtests' is the -Q switch. I have a -q which doesn't print test failures while tests are running, but as you can see, one of my 'stress tests' caused a problem: TAPx-Parser $

Re: TAP::Tests

2007-01-05 Thread Ovid
--- Ovid [EMAIL PROTECTED] wrote: Some of the limitations of TAPx::Parser are due to how Test::Builder works. One thing which isn't making it into 'runtests' is the -Q switch. I have a -q which doesn't print test failures while tests are running, but as you can see, one of my 'stress

Re: Thoughts about test harness summary

2007-01-05 Thread David Golden
On 1/5/07, Ovid [EMAIL PROTECTED] wrote: Moreover, it looks really horrid with non-monospaced fonts. You use non-monospaced fonts in your terminal? :) Thta's gmail for you. David

Re: TAP::Tests

2007-01-05 Thread Michael G Schwern
Ovid wrote: Some of the limitations of TAPx::Parser are due to how Test::Builder works. One thing which isn't making it into 'runtests' is the -Q switch. I have a -q which doesn't print test failures while tests are running, but as you can see, one of my 'stress tests' caused a problem:

Re: TAP::Tests

2007-01-05 Thread Ovid
--- Michael G Schwern [EMAIL PROTECTED] wrote: TAPx-Parser $ /usr/bin/perl -Ilib bin/runtests -qm tbad/ tbad/060-aggregator..ok tbad/badtestsFAILED tests 1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19, 20, 22, 23, 25, 26, 28, 29, 31, 32, 34, 35, 37, 38, 40, 41, 43, 44,

Re: TAP::Tests

2007-01-05 Thread Michael G Schwern
Ovid wrote: --- Michael G Schwern [EMAIL PROTECTED] wrote: TAPx-Parser $ /usr/bin/perl -Ilib bin/runtests -qm tbad/ tbad/060-aggregator..ok tbad/badtestsFAILED tests 1, 2, 4, 5, 7, 8, 10, 11, 13, 14, 16, 17, 19, 20, 22, 23, 25, 26, 28, 29, 31, 32, 34, 35, 37, 38, 40, 41,

Re: TAP::Tests

2007-01-05 Thread Ovid
--- Michael G Schwern [EMAIL PROTECTED] wrote: That list of FAILED tests does not come from Test::Builder. I'm still missing something. You are correct. I had bollixed my tests (it turns out that running tests which run tests and then drive the results through the test harness I'm testing is

First try at sample test output.

2007-01-05 Thread Ovid
OK, here's a first pass at sample test output with my new test harness. Note that the -q option is enabled to suppress a very long test output. Let me know what you think. I do realize that the indentation on the failure results might still cause problems with non-monospaced displays, but I

Re: Thoughts about test harness summary

2007-01-05 Thread Paul Johnson
On Fri, Jan 05, 2007 at 01:50:54PM -0600, Andy Lester wrote: On Jan 5, 2007, at 1:46 PM, Ovid wrote: Failed Test Total Fail List of FailedTODO Passed -- t/bar.t 13 9 2, 6-8, 13, 17, 33-35 3-4