Re: Thoughts about test harness summary

2007-01-05 Thread Paul Johnson
On Fri, Jan 05, 2007 at 01:50:54PM -0600, Andy Lester wrote: > On Jan 5, 2007, at 1:46 PM, Ovid wrote: > > >Failed Test Total Fail List of FailedTODO Passed > >-- > >t/bar.t 13 9 2, 6-8, 13, 17, 33-35 3

First try at sample test output.

2007-01-05 Thread Ovid
OK, here's a first pass at sample test output with my new test harness. Note that the -q option is enabled to suppress a very long test output. Let me know what you think. I do realize that the indentation on the failure results might still cause problems with non-monospaced displays, but I ca

Re: TAP::Tests

2007-01-05 Thread Ovid
--- Michael G Schwern <[EMAIL PROTECTED]> wrote: > That list of "FAILED" tests does not come from Test::Builder. I'm > still missing something. You are correct. I had bollixed my tests (it turns out that running tests which run tests and then drive the results through the test harness I'm testin

Re: TAP::Tests

2007-01-05 Thread Michael G Schwern
Ovid wrote: > --- Michael G Schwern <[EMAIL PROTECTED]> wrote: >>> TAPx-Parser $ /usr/bin/perl -Ilib bin/runtests -qm tbad/ >>> tbad/060-aggregator..ok >>> tbad/badtestsFAILED tests 1, 2, 4, 5, 7, 8, 10, 11, 13, >> 14, >>> 16, 17, 19, 20, 22, 23, 25, 26, 28, 29, 31, 32, 34, 35, 37,

Re: TAP::Tests

2007-01-05 Thread Ovid
--- Michael G Schwern <[EMAIL PROTECTED]> wrote: > > TAPx-Parser $ /usr/bin/perl -Ilib bin/runtests -qm tbad/ > > tbad/060-aggregator..ok > > tbad/badtestsFAILED tests 1, 2, 4, 5, 7, 8, 10, 11, 13, > 14, > > 16, 17, 19, 20, 22, 23, 25, 26, 28, 29, 31, 32, 34, 35, 37, 38, 40, > 41,

Re: TAP::Tests

2007-01-05 Thread Michael G Schwern
Ovid wrote: > Some of the limitations of TAPx::Parser are due to how Test::Builder > works. One thing which isn't making it into 'runtests' is the -Q > switch. I have a -q which doesn't print test failures while tests are > running, but as you can see, one of my 'stress tests' caused a problem: >

Re: Thoughts about test harness summary

2007-01-05 Thread David Golden
On 1/5/07, Ovid <[EMAIL PROTECTED]> wrote: > Moreover, it looks really horrid with non-monospaced fonts. You use non-monospaced fonts in your terminal? :) Thta's gmail for you. David

Re: TAP::Tests

2007-01-05 Thread Ovid
--- Ovid <[EMAIL PROTECTED]> wrote: > Some of the limitations of TAPx::Parser are due to how Test::Builder > works. One thing which isn't making it into 'runtests' is the -Q > switch. I have a -q which doesn't print test failures while tests > are > running, but as you can see, one of my 'stress

TAP::Tests

2007-01-05 Thread Ovid
Some of the limitations of TAPx::Parser are due to how Test::Builder works. One thing which isn't making it into 'runtests' is the -Q switch. I have a -q which doesn't print test failures while tests are running, but as you can see, one of my 'stress tests' caused a problem: TAPx-Parser $ /usr/b

Re: Thoughts about test harness summary

2007-01-05 Thread Andy Lester
On Jan 5, 2007, at 1:49 PM, Ovid wrote: Moreover, it looks really horrid with non-monospaced fonts. You use non-monospaced fonts in your terminal? :) Results might get emailed to someone, or put into a web page that doesn't do monospaced fonts. SourceForge's forums are an excellent exa

Re: Thoughts about test harness summary

2007-01-05 Thread Andy Lester
On Jan 5, 2007, at 1:46 PM, Ovid wrote: Failed Test Total Fail List of FailedTODO Passed -- t/bar.t 13 9 2, 6-8, 13, 17, 33-35 3-4 t/foo.t 10 10 5, 19, 27, 37-38, 117 9-11 Did you mean

Re: Thoughts about test harness summary

2007-01-05 Thread Ovid
--- David Golden <[EMAIL PROTECTED]> wrote: > On 1/5/07, Andy Lester <[EMAIL PROTECTED]> wrote: > > > > On Jan 5, 2007, at 1:28 PM, Nicholas Clark wrote: > > > > >> Failed Test | Total | Fail | List of Failed | TODO Passed > > >> > +---+--++---

Re: Thoughts about test harness summary

2007-01-05 Thread Ovid
--- Nicholas Clark <[EMAIL PROTECTED]> wrote: > I'd like Wstat, even if I don't have Stat. I like to know if tests > coredumped. > I may be in a minority here, but being able to optionally switch to > that output is useful. Fair enough. > I like the prominence of TODO passed Cool. I hated wedg

Re: Thoughts about test harness summary

2007-01-05 Thread David Golden
On 1/5/07, Andy Lester <[EMAIL PROTECTED]> wrote: On Jan 5, 2007, at 1:28 PM, Nicholas Clark wrote: >> Failed Test | Total | Fail | List of Failed | TODO Passed >> +---+--++ >> t/bar.t | 13| 4|2, 6-8 |3-4 >>

Re: Thoughts about test harness summary

2007-01-05 Thread Andy Lester
On Jan 5, 2007, at 1:28 PM, Nicholas Clark wrote: Failed Test | Total | Fail | List of Failed | TODO Passed +---+--++ t/bar.t | 13| 4|2, 6-8 |3-4 +---+--++-

Re: Thoughts about test harness summary

2007-01-05 Thread Nicholas Clark
On Fri, Jan 05, 2007 at 11:11:25AM -0800, Ovid wrote: > Pursuant to some discussion with BrowserUK at > http://perlmonks.org/?node_id=593087, I'm looking at this and seeing > some problems. > > -- > > Failed Test Stat Wstat Total Fail List of Failed >

Thoughts about test harness summary

2007-01-05 Thread Ovid
Pursuant to some discussion with BrowserUK at http://perlmonks.org/?node_id=593087, I'm looking at this and seeing some problems. -- Failed Test Stat Wstat Total Fail List of Failed - t/bar.t4 1024134 2 6-8 t/foo.t

Re: Comment about BAIL_OUT

2007-01-05 Thread Michael Peters
Greg Sabino Mullane wrote: > Michael G Shwern wrote: >> Such a bother. >> ... >> You can even get clever and pack the setup/teardown calls into >> loading the module so you have even less code per script. >> >> Now each test runs independently and cleans itself up. > > True, but at the expense

Re: Comment about BAIL_OUT

2007-01-05 Thread Greg Sabino Mullane
Michael G Shwern wrote: > Such a bother. > ... > You can even get clever and pack the setup/teardown calls into > loading the module so you have even less code per script. > > Now each test runs independently and cleans itself up. True, but at the expense of having to run the startup and cleanup

Re: Desired test output?

2007-01-05 Thread Eric Wilhelm
# from Ovid # on Friday 05 January 2007 01:50 am: >TAPx::Parser collects far more >information than Test::Harness, so if there's more stuff you'd like to >see, that's fine, too. You could dump it all into some kind of data (yaml?) file, then execute $ENV{TAP_RESULTS_VIEWER} or something? TAP_RE

Desired test output?

2007-01-05 Thread Ovid
>From http://www.perlmonks.org/?node_id=593087: I'm working on TAPx::Parser and trying very hard to make my TAPx::Harness output as similar to Test::Harness output as is feasible. I am doing this primarily because if the basic output is too different from what folks are used to, the "strangeness"

Re: Comment about BAIL_OUT

2007-01-05 Thread Michael G Schwern
Ovid wrote: > However, if you use the '-s' switch to shuffle your tests and bailout > is not first, then some tests will run until the BAIL_OUT is hit. This > seems to violate the principle that tests should be able to run in any > order without dependencies. It doesn't violate the principle sinc

Re: Comment about BAIL_OUT

2007-01-05 Thread Michael G Schwern
Greg Sabino Mullane wrote: > [1] I've never had a need for random tests myself. The only reason I > break mine apart is to isolate testing various sub-systems, but I almost > always end up having some dependencies put into an early "00" file. I > also tend to a have a final "99" cleanup file. While

Re: Comment about BAIL_OUT

2007-01-05 Thread David Landgren
Greg Sabino Mullane wrote: [...] [1] I've never had a need for random tests myself. The only reason I break mine apart is to isolate testing various sub-systems, but I almost always end up having some dependencies put into an early "00" file. I also tend to a have a final "99" cleanup file. Whi