Re: Test::Harness wrangling

2006-06-29 Thread chromatic
On Thursday 29 June 2006 07:26, Michael Peters wrote: > This is a general problem with the way some of the testing modules work. > They output their diagnostic messages as comments to STDERR. They should > send them instead to STDOUT so that it can be synced. TAP does support > comments, so I'm no

Re: Update from YAPC::NA

2006-06-29 Thread Ovid
Hi Adam, I definitely realize that some of the feature requests do not seem like bugs, but if some are easy to implement then I would think they should be worthy of consideration. David Wheeler's complaint about unsynchronized test output is definitely a bug and it's one I've been bitten by ma

Re: Test::Harness wrangling

2006-06-29 Thread David Wheeler
On Jun 29, 2006, at 07:26, Michael Peters wrote: This is a general problem with the way some of the testing modules work. Only because they all use Test::Builder's output methods. This doesn't resolve the problem of non-test modules emitting things to STDERR that could be useful when track

Update from YAPC::NA

2006-06-29 Thread Adam Kennedy
I noticed a lot of people weighed in with non-bug posts to Andy's bug fix request. I just wanted to note that Wednesday we had a PITA/Test::Builder/Testing BOF at which a bunch of people had an hour long talk about the structure of Test::et-al TAP and various other topics. I think we addressed

Re: Test::Harness wrangling

2006-06-29 Thread Michael Peters
David Wheeler wrote: > On Jun 28, 2006, at 21:13, Andy Lester wrote: > >> Any bugs that we especially need to work on? > > This might be Test::Builder, but I've seen many times on my Mac where > STDERR and STDOUT output is out of sync. For example, info about why a > test failed can be miles fr

Re: Test::Harness wrangling

2006-06-29 Thread David Wheeler
On Jun 28, 2006, at 21:13, Andy Lester wrote: Any bugs that we especially need to work on? This might be Test::Builder, but I've seen many times on my Mac where STDERR and STDOUT output is out of sync. For example, info about why a test failed can be miles from the line where the "no ok" a

Re: Test::Harness wrangling

2006-06-29 Thread Dominique Quatravaux
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ovid wrote: > sub some_test_case : Test(3) { my $test = shift; # same as $self, > actually ok some_func(); ok another_func(), 'awesome'; is $answer, > 42; } Also for those of us who are fed up with counting tests manually: sub some_test_case : Tes

Re: Test::Harness wrangling

2006-06-29 Thread Nicholas Clark
On Wed, Jun 28, 2006 at 11:13:02PM -0500, Andy Lester wrote: > Tomorrow, Adam Kennedy and I (and Schwern?) will be banging on > Test::Harness. > > Any bugs that we especially need to work on? They're not so much bugs as structural annoyances that it would be useful to relax to make subclassing

Re: Test::Harness wrangling

2006-06-29 Thread Ovid
- Original Message From: Michael Peters <[EMAIL PROTECTED]> > Not a bug, but a feature request. Abstract out the TAP parsing into a separate > module so that it can be done by more than just Test::Harness::* modules. I started that, but I hit three snags. 1. It would be very useful to

Re: Test::Harness wrangling

2006-06-29 Thread Ovid
- Original Message From: Luke Closs <[EMAIL PROTECTED]> > Maybe slightly unrelated, but have you guys considered adding some > extra (optional) meta info to the TAP spec? For instance at $work, > we group our individual assertions into higher level tests like this: > > # TITLE: Some

Re: Test::Harness wrangling

2006-06-29 Thread Ovid
If it's not too late, I would really love to see the TAP output disambiguate between diag() output and test failure output. I think it was Adrian who suggested something as simple as: not ok 52 - forced failure # Failed test 'forced failure' # in t/db_version.t at line 225. # got:

Re: Test::Harness wrangling

2006-06-29 Thread Andy Lester
On Jun 29, 2006, at 5:21 AM, Randy W. Sims wrote: Tomorrow, Adam Kennedy and I (and Schwern?) will be banging on Test::Harness. Finalize Test::Harness::Straps. That is THE reason we're doing this. Everything else is gravy. -- Andy Lester => [EMAIL PROTECTED] => www.petdance.com => AIM:pe

Re: Test::Harness wrangling

2006-06-29 Thread Shlomi Fish
On Thursday 29 June 2006 13:27, Michael Peters wrote: > Andy Lester wrote: > > Tomorrow, Adam Kennedy and I (and Schwern?) will be banging on > > Test::Harness. > > > > Any bugs that we especially need to work on? > > Not a bug, but a feature request. Abstract out the TAP parsing into a > separate

Re: Test::Harness wrangling

2006-06-29 Thread Shlomi Fish
On Thursday 29 June 2006 07:13, Andy Lester wrote: > Tomorrow, Adam Kennedy and I (and Schwern?) will be banging on > Test::Harness. > > Any bugs that we especially need to work on? > Well, giving my angle as the Test::Run maintainer (which is a fork of Test::Harness): 1. I may have eliminated s

Re: Test::Harness wrangling

2006-06-29 Thread Michael Peters
Andy Lester wrote: > Tomorrow, Adam Kennedy and I (and Schwern?) will be banging on > Test::Harness. > > Any bugs that we especially need to work on? Not a bug, but a feature request. Abstract out the TAP parsing into a separate module so that it can be done by more than just Test::Harness::* m

Re: Test::Harness wrangling

2006-06-29 Thread Randy W. Sims
Andy Lester wrote: Tomorrow, Adam Kennedy and I (and Schwern?) will be banging on Test::Harness. Finalize Test::Harness::Straps. Make it possible to programatically run tests and to capture output. Eg. Module::Build has need to run tests, displaying output as normal, but to also capture that

Re: Test::Harness wrangling

2006-06-29 Thread Michael Peters
Luke Closs wrote: > Maybe slightly unrelated, but have you guys considered adding some extra > (optional) meta info to the TAP spec? For instance at $work, we group > our individual assertions into higher level tests like this: > > # TITLE: Some test case > ok 1 > ok 2 - awesome > not ok 3

Re: Test::Harness wrangling

2006-06-29 Thread Luke Closs
Maybe slightly unrelated, but have you guys considered adding some extra (optional) meta info to the TAP spec? For instance at $work, we group our individual assertions into higher level tests like this: # TITLE: Some test case ok 1 ok 2 - awesome not ok 3 I believe we also have an ACTI